This week, I’ve had to make some changes to an automation we had setup for a customer in Workato. The original recipe was made by a co-worker with a bunch of javascript nodes. Even though I’m customer facing and generally not considered a technical employee, I knew the engineer who worked on this was swamped, so I decided to jump in.

In order to greatly reduce the number of nodes needed in the single recipe, I created a list of dictionaries in python (with one of the values being another list!) and a quick for loop to grab all the appropriate elements from the various dictionaries. I had thankfully already copied all the JS nodes to a single file for easy reference and retrieval. That one file had over 8,000 lines, and I needed to replace all the javascript elements with python.

Here’s one node of Javascript that I was working with:

// @param input fields supplied in the recipe step, as an object
// @return object matching the output schema
// Eg: Code for returning time zone for an IP address

exports.main = async ({ user_email }) => {
  let all_domains = [
 'domain_one',
 'domain_two',
 'domain_three',
 'domain_four',
 'domain_five',
 'domain_six',
]

  let user_domain = user_email.slice(user_email.indexOf('@'));
  let domain_found = all_domains.indexOf(user_domain) > -1;
  return { domain_found };
}

Most nodes were very similar, all except that their all_domains array had anywhere from 1 to 500+ elements. Thanks to the neovim plugin Telescope Cmdline by Jon Arrien, you can very easily see your past history of Neovim commands. I’ve gone back through and pulled out a few special ones that helped and that I was very surprised how well they worked. I don’t claim that this was the fastest or best way to clean up a large file with repeating functions, but I definitely learned a bunch about searching-and-replacing in Neovim, and any future needs for this will be way faster! One thing that has been really helpful with Neovim is that as you run the first portion of all the commands below, Neovim should highlight what it finds. This is really helpful for real-time debugging of your regex.

Commands and Quick Explanations

Command: :g/\//+2/d

Explanation: The g in the command stands for global (hint, run :h :g in vim!) and will search across the entire buffer you have open. In this command I’m looking for all Javascript single-line comments. Since the forward slash is a special character for neovim, you need to escape it with a backwards slash, hence the odd looking \/. The second forward slash is the next parameter neovim is looking for, and by adding +{num}, you can search for the pattern plus a number of lines. Closing out the command (again, next parameter comes after the forward slash) with d for delete will then delete everything with you searched for, plus the additional number of lines.

Bonus: if you wanted to make sure that you only pulled lines that began with a comment, you’d add a carrot to the beginning of the command. :g/^\//+2/d

Command: :%s/let all_domains = /"domains" : /g

Explanation: Moving on from g to s, the s here stands for substitute. There’s one big difference in using g and s, though. While g will search globally, a singular s will only search on your current line. To search the entire file/buffer, you need to include the percent character before the s. The rest of this is fairly straight forward, I am searching for the declaration of a javascript variable and then substituting it with a python dictionary key. Since both javascript and python declare arrays with square brackets, I wanted to make sure I kept that after the substitution. Lastly, unless you only want the subsitute to change the first occurrence of what it finds, you’ll need to close out the command with the g flag. Neovim’s docs say:

[g] Replace all occurrences in the line.  Without this argument,
 replacement occurs only for the first occurrence in each line.  If the
 'gdefault' option is on, this flag is on by default and the [g]
 argument switches it off.

Command: :%s/^exports.main.*/{ "uuid" :/g

Explanation: Really similar to the above command, but in this case, there are no characters I needed to save post-substitution. We already know what %s does, so the pattern I’m looking for is all lines that start with exports.main. Since I don’t need anything after that start of the line, I included .* which looks for any character until the end of the line. After the pattern I write out what I need to replace it with, which is the first portion of the dictionaries I need. Finally, we close it out with a global change so it changes it everywhere.

Command: :%s/return.*/], Explanation: Finally, I used this one to close out the domains list/array that I’m using.

Bonus Command: %s/"props": \[\(.*\)\]\,/"props" : \1,/g

Explanation: I won’t go over all the characters and regex commands as I’m reusing a lot of them from the above commands. What is different here is that I’ve lumped the wild card in parenthesis like so: (.*\). Now, I can call back what is pulled by using \1. The command above is looking for a dictionary item with a list as the value. I didn’t need the list, just a string, since all values in my list of dictionaries are single values. By searching for "props": [ string ], and using the \1, I’m able to keep the string intact and the result is "props": string,.

This was the milestone command to learn for me and helped unlock so much potential in my brain for other use cases. For all the other commands I’ve gone over above, I’m fairly simply just replacing characters I no longer need that exist either before or after characters I do need. Now that I can keep strings intact for substitutions, I bet I could revisit all the above commands and make them even more efficient.

What I’d like to learn next is if I can use an “or” statement in the regex. Let’s say I have the same situation as above - a list of dictionaries with some dict values being a list. Well, if the lists are all formatted differently, some on new lines, others on a single line, can I select both in a single command? Something like: :%s/"domains": \[\(.* OR \n\)/.

That will be a post for another time! Let me know on Mastodon if you have any other Neovim search and replace tips and tricks that have become invaluable to your workflow. I’m always amazed and impressed with this community and how differently people are able to use the same tool.

By the way, here’s the final output of running the above commands on the sample code above, converted to Python.

mappings = {
  'uuid': "1234-1234-1234-1234",
  'domains': [
     'domain_one',
     'domain_two',
     'domain_three',
     'domain_four',
     'domain_five',
     'domain_six',
    ],
  'props': 'property-mapping',
}

Resources

Here are some various links of resources that I found helpful when learning about the above commands. I’m assuming you’ve already read through anything in Neovim’s Helpdocs (:h), but sometimes you need a different take and explanation.