I’m a front-end developer, and I keep my front-end projects in OneDrive. Due to how Node.js organizes dependencies, each project has a node_modules directory with tens of megabytes of tiny files which take forever to sync.
What can I do to avoid synchronizing node_modules directories while keeping the projects in OneDrive?
I know this is an older question, but I ran into the same issue. The solution I came across was to first create an empty node_modules folder and then sync that with the cloud.
Once the empty folder was synced, I then went into
OneDrive settings → Choose folders
and deselected the node_modules folder.
Then when you run npm install in the project root folder, OneDrive will detect a conflict with the contents of the folder in the cloud. It will not sync.
The downside of this approach is that OneDrive produces sync errors, but it seemed to work well for my needs.
OneDrive doesn’t allow to exclude directories by name, and that’s something that won’t change soon:
[OneDrive’s representative response on a UserVoice request:] Not right now
Some people recommend to uncheck directories you don’t want to sync in OneDrive settings, but that became impossible with OneDrive Files on-demand.
However, you can solve this by tweaking things on the Node.js’ side. Make the node_modules directory a file symlink to a different place:
# Open any place outside OneDrive
cd D:\node_dependencies
# Make a symlink target. After linking, node_modules for your project
# will be installed here
mkdir node_modules_for_my_project
# Open the project directory
cd <the project directory>
# Make a *file* link from node_modules to the newly created directory.
# You will need to delete existing node_modules directory if it exists
cmd /C "mklink node_modules D:\node_dependencies\node_modules_for_my_project"
The important piece here is that you create a file symlink, not directory one. OneDrive won’t recognize and sync this symlink, whereas Node.js will work with it as intended:
Tested with OneDrive v17.3.7101.1018 and OneDrive Files on-demand enabled.
This is not a universal solution. A drawback of this is that Explorer, Powershell and other tools won’t recognize node_modules as a directory:
However, Node.js-based code editors will read it just fine:
I've found that the best solution is creating a 'local' folder outside the OneDrive folder where the project will be saved and any modules will be installed. This is the actual project.
To sync it to OneDrive I manually create a 'parallel' folder and copy only the content I want using 'RoboCopy'. You can then schedule this task.
The files & folders names/wildcards to ignore were copied from the .gitignore file
The solution is based on a few articles:
Schedule RoboCopy task and simple examples
RoboCopy documentation by Microsoft
you can also use GUI to create the .bat task file to be scheduled. GUI by microsoft & ChoEazyCopy GUI which is much better
For Typescript users out there, add node_modules to your path to avoid module resolution issues.
In tsconfig.json add :
"paths": {
[... some paths ...],
"*": [
"node_modules/*"
]
}
Edit: Still have some issues. I can build the project but ts-node and nodemon fire Cannot find module errors. Well, this is still an issue then :(
After reading these proposed options, I ended up deciding to put everything outside of OneDrive and create a script with Robocopy to backup to OneDrive whenever there was a change in the projects folder.
Something like:
robocopy SOURCE DEST /mir /xd node_modules
Just delete the files or folders in OneDrive that you don't want to sync , they're still there in your local disk.
I use a development folder that OneDrive doesn't back up and a backup folder OneDrive does back up and copy the development folder to the backup folder with a batch file. The following is essentially an example / step-by-step of @RanST's answer.
@ECHO ON
ECHO Copy started.
robocopy "{{SOURCE}}\PROJECTS" "{{DESTINATION}}\PROJECTS" /xd node_modules .serverless /s /z /r:3 /w:10 /tbd /np /ts
ECHO Copy completed!
PAUSE
/xd = exclude. Simply add the folder names to exclude.
/s = include subfolders exclude empty directories.
/z = retryable. /r:3 = 3 retries. /w:10 = wait 10 secs between retries.
/tbd = wait for a share names if they caused a retry error
/np = don't display progress
/ts = add a timestamp to the log
The Microsoft Documentation explains all the options.
I wanted a backup that didn't need to be updated every new project, so I chose to back up a parent folder and maintain the file structure when I created the development specific folders.
Main synced folder
/MY_WHOLE_BACKUP/.../PROJECTS/clients/client1/parentProject/subProject/root/myApp/app.js
DON'T simplify the path
/MY_LOCAL_FILES/.../PROJECTS/client1/myApp/app.js
DO keep it all
/MY_LOCAL_FILES/.../PROJECTS/clients/client1/parentProject/subProject/root/myApp/app.js
Search for and open the Task Scheduler in Windows.
Click Create Task....
Name the task.
Click the Triggers tab.
Click the New... button.
Adjust the schedule to your preference.
Click OK.
Click the Actions tab.
Click the New... button.
Browse for your batch file.
Click OK.
BOOM!
Test and configure as needed. I hope this helps.
I have found that using dev containers can fix this issue if you can put the time into creating your environment.
