Skip to main content

Search and Replacement Techniques

From time to time it’s a good idea to turn back to the basics. Even after years of Emacs usage you will potentially find things you overlooked before - or new ways established without you noticing.

This week I searched for workflows to find and replace matches with Emacs. In this post I will summarize the different techniques I found and hopefully provide you with some new methods you might want to try or incorporate into your own workflow. I will contrast "the old way" with the "modern way" in each section, but notice that by modern I don’t necessarily mean better.

"Old" just means that the method has been built-in for a long time and "modern" means, external packages or newer additions to stock Emacs are used to achieve the task. As Emacs is big and the community even bigger the described methods are not near complete. Feel free to add you own suggestions in the comments.

TL;DR

My current favorites are:

  • objed to replace the symbol at point in current defun or buffer

  • anzu to replace a regex in current buffer

  • project (built-in) to replace a regex or symbol in a project

  • swiper and counsel, deadgrep to search and narrow down matches in a buffer or project

In buffer search and replace

The old way (isearch, query and replace, occur and keyboard-macros)

If you are new to Emacs and want to learn more about isearch I highly recommend this nice tutorial by Bastien Guerry.

To start a classic search and replace session in Emacs jump back to the beginning of the buffer with M-< and start a query replace with M-% or C-M-% (regex version) or the non-interactive versions M-x replace-string and M-x replace-regexp.

The most useful keys inside a query replace sessions are SPC to replace and move to next match, n to skip a match, ! to replace all remaining matches in the currrent file, Y to replace all matches in all upcoming files, N to skip the rest of matches in current the file.

If you ever want to repeat a replace command with slightly different input try C-x ESC ESC to get the last minibuffer command as an Elisp expression, although using the minibuffer history using M-p or M-n is usually enough.

Another neat thing about query replace in Emacs, which you will not find in other Editors, is that you are allowed to use Elisp on the matched groups of the regex. For example you could use this to replace matches for "movie" with "film" and matches for "movies" with "films" using the search term movie\(s\)? and \,(if \1 "films" "film") as the replacement expression. Another common use case is to transform numbers in the matches using the format function.

If you are in the middle of a search and want to replace the used search term in the whole buffer, there is one more built-in method I would like to describe here. Namely you can invoke occur (with your current search term) using M-s o. This will give you a complete listing of the matched lines. Now jump to the first match and start recording a keyboard macro with F3 or C-x (, after editing the match to your liking jump to the next match with M-g n. Finally stop the recording of the macro with F4 or C-x ).

This will allow you to execute the macro with C-x e or F4. To execute until you reach the end of the buffer. You can use a zero prefix argument C-0 C-x e to repeat your edits on all the matches found.

Update: As hmelman points out you can use e in occur buffers to enter occur-edit-mode and after editing commit the changes with C-c C-c. This even works for multi-occur!

One final built-in technique that deserves a mention here, is that you can always restrict your replacements by narrowing your buffer before you start using one of the methods above. For example mark a paragraph with M-h and narrow to its region with C-x n n. To exit the narrowing use C-x n w to widen the buffer to its old size again.

The modern way (anzu, swiper, helm, wgrep, mc, iedit, symbol-overlay)

The "old way" using isearch can be improved by using the anzu package. It shows you the regex matches and regex groups highlighted in the buffer. Another package that does this is visual-regexp. The replacements are shown inline, too. This is very helpful for constructing the right search and replacement terms. Another neat feature of anzu is that it shows you the number of matches in the mode line.

Many people use packages which show interactive previews of the search matches like swiper or helm. For both of these you can switch to an editable buffer of the matches. Using helm-swoop you can press C-c C-e to switch to edit mode and apply changes to original buffer by C-x C-s. For swiper you press C-c C-o to switch to the ivy-occur buffer. There you press C-x C-q to edit it using wgrep and C-x C-s to commit your changes.

The way you edit the matches is totally up to you. You can start a simple query replace session as described in the last section or go fancy using multiple-cursors, iedit or symbol-overlay. Those three packages can be used to quickly replace the symbol at point, too. If that’s all you want to do, using one of these in the first place is a bit faster and more convenient for this task.

My way

I like to use swiper to get an overview of matches, but every now and then isearch is handy, too. For example I like to use it for replacing matches in the ivy-occur buffer. To quickly navigate and replace symbols in the current buffer or defun I use my own package called objed and for replacing a regexp I use anzu.

Search and replace across multiple files or buffers

There are different ways to solve this task. Either you collect a list of files first and then run a query replace on them or you immediately collect matches in files of interest and edit the matches afterwards. Depending on the used method the set of files can be specified by a glob pattern, a regex, a TAGS file or special project files (.gitignore, .project).

The old way

Find files/matches

To narrow to the set of files you are interested in you have several options:

  • M-x find-name-dired will prompt you for a root directory and a filename pattern, the results are collected in a dired buffer

  • M-x dired will prompt you for the directory. Mark all the files with t or to narrow to specific files, first insert sub directories using C-u i (add "-R" to recurse) and mark files matching a regex using % m

  • Create a TAGS file for your project. For Elisp you can generate one using M-! find . -name '*.el' -print | etags -. To append other files for example add C files to the table use find . -name '*.[ch]' -print | etags --append -. Many other languages are supported see M-x man etags.

To collect a list of matches in a set of files you can use one of the following:

  • M-x rgrep will collect lines matching a glob pattern in matched files in a *grep* buffer.

  • M-x vc-git-grep will collect lines matching a glob pattern for files in your git project and respects your .gitignore.

Replace matches
  • If you went the M-x dired route use Q and you will be prompted for query/substitution afterwards. This will start a regular query replace session which will run through all marked files.

  • If you created a TAGS file you can now use M-x tags-query-replace to run query replace on all tags in your TAGS file.

  • When you used M-x rgrep or M-x vc-git-grep you could define a keyboard macro like described above.

The modern way (swiper, wgrep, helm, projectile, project-find-regexp)

M-x rgrep, vc-git-grep and the various counsel commands like counsel-rg/ag/pt/git-grep can be used in conjunction with wgrep the same way as described in the previous section.

For helm there are similar methods using M-x helm-projectile-grep, helm-occur or helm-do-grep.

Using projectile and projectile-replace, projectile-replace-regexp you can run an interactive query replace on project files, too. It uses tags-query-replace under the hood.

With Emacs 25 comes a new built in project library. Using project-find-regexp you can search in the current project which recognizes various version control types automatically. This pops up a buffer where you can navigate the matches with n, p and start an interactive query replace on the matches with r.

My way

To replace matches I prefer project-find-regexp which is convenient to use and built-in.

For interactive file search and grep I use swiper and counsel-rg again and sometimes the neat deadgrep package.

Closing words

The methods described often don’t save the changed files. To save the changes you made to all buffers you can either use C-x s followed by ! or call ibuffer and mark all unsaved buffers with * u followed by S to save them. Using ibuffer has the benefit that you can clean up by closing all marked files afterwards with D.