Solving painful syncing of node_modules when using Dropbox (or Google Drive)

Developing for the web and having smart up-to-date syncing between multiple computers has improved quite a bit over the course of last several years. Thanks to the improvements to Dropbox, Google Drive and other services working across several computers with the same code base and files have improved by leaps and bounds.

Most developers use Github or Bitbucket repositories to sync codebases across computers and servers however these services are not meant for syncing larger files such as Photoshop or many other types.

As someone who works at home on a desktop Mac and my Mac Book Pro when I’m on the go and travel, I needed a good solution to sync my projects, fonts, designs and other things seamlessly between the computers without doing much work and having it automated so I can simply work on my desktop computer and at the moment’s notice pick up my laptop and continue where I left off.

Using Google Drive or Dropbox works great for this, setting up syncing on both machines, however one of the biggest problems when trying to sync my development files was the complete lack of “filtering” or something like .gitignore to any of these services.

One of the main pain points in using Dropbox for code are little small folders called node_modules and composer’s vendor. These folders caused me so many problems syncing because they usually contain 10s of thousands of files and syncing them sometimes causes a complete mess with services like Dropbox or Google Drive taking hours or even days to sync everything (if I’m lucky) if I removed the node_modules files or changed them things just keep getting messy (not to mention it kills your computers). This made working across computers with these services a nightmare.

Up to this point, I have solved a lot of little things in my workflow when it comes to syncing fonts for example, and not only synced but activated across computers thanks to a little program called RightFont an intelligent and a really cool font manager that deals with live sets by watching specific folders. This feature is called Live Lists. Needless to say, I have my fonts on Dropbox synced and that font folder is synced on both machines so if I add a font on one computer, the Dropbox would sync and the other computer’s RightFont would notice the new fonts synced and activate them automatically. Works beautifully. Never again I have to worry if I’m missing some specific font between computers and waste time installing them.

Now I was determined to find an elegant solution in syncing my code across computers in a similar way, but making sure that node_modules and vendor folders are not uploaded/sync-ed with dropbox because these folders don’t necessarily change that often, and even if they do, my package.json file would still be sync-ed and all I have to do is run npm update and my current my machine would update node_modules or vendor folders (composer update). Simple enough.

One problem though, neither Dropbox or Google Drive have that type of filtering in the same way as Git has.

By looking at a lot of solutions I have found a really cool application called GoodSync. This application does automated replication/syncing on your computer between folders based on rules and most importantly it has filters you can define on Including/Excluding folders or files filters.

So, I’ve rolled up my sleeves and set to make this work.

I have installed Dropbox on both my desktop Mac and MBP. Dropbox automatically creates Dropbox folder that gets sync-ed as files change, so in order to avoid issues I had before, I figured working on local files in a Documents folder for example would be my setup but then have GoodSync replicate/sync the structure with the local Dropbox folder on that machine (usually on my external hard drive) but applying the exclude filters where I would omit node_modules and vendor folders. This way my files between the Documents working folder, and local Dropbox folder would be updated in real-time as soon as I hit save on a file (GoodSync pretty much works the same way Dropbox does for the cloud but on your local folders), as GoodSync finished it’s process of syncing, Dropbox would then sync the files from Dropbox folder to the cloud. This all works surprisingly well and fast.

This allows me to have node_modules and vendor folders excluded from going up to Dropbox but I still have them locally on the machine. When I switch to another computer, the files are all synced and I would just run npm update or/and composer update and those 2 folders would get updated very quickly and I would keep working smoothly without skipping a beat.

With this working, now all files that are changed on Dropbox would be downloaded from the cloud into Dropbox folder, and GoodSync would then automatically sync to my Documents folder as well making the process work both ways seamlessly.

Setup like this finally solved my problems with syncing and working between multiple computers and having everything up to date via Dropbox but without the pain of syncing thousands of files from node_modules that slow down this process to unusable. GoodSync has full automated service running in the background so I get to concentrate on work and not worrying about syncing and working on multiple computers.

Hopefully this helps someone who has similar goals in automating their cross-computer development and sync-ing.