Tired of waiting for `npm install`
every time you touch Docker?
It’s no secret that ‘npm install' can take a long time. An excessive amount of time, even.
When you have 30+ modules that each install 30+ modules, which install 30+ modules, that each install … ad-infinitum … it’s a wonder that your computer has any hard drive space left over.
And the amount of time it takes? Well, as Sergio Rodrigo recently said on Twitter:
“Life is that thing that passes while you wait for npm install to finish.”
Now add Docker into the equation – with a host-mounted volume so you can edit your code, of course – and it only gets worse.
Docker, for all of its tremendous benefits, seems to make the installation of node modules a regular event. In a given day, ‘npm install’ can be run dozens of times, often several times per hour when you’re working on your Dockerfile or standing up a development container.
And having a host-mounted volume for editing code doesn’t help this situation. You're not only running npm install frequently, but doing so on a folder that is notoriously slow – nearly half the speed of a folder without a host mounted volume.
It’s easy to see why the npm delay is eating your lunch with all of this happening. What was once an already slow install at 5 minutes is now more like 8 to 10 minutes.
Even the Markov bots on Twitter realize the potential for melting your laptop with all the work your CPU and hard drive must do in this case!
So, Why Does It Take So Long?
There’s no doubt that Docker offers a tremendous amount of value in development, testing, deployment and beyond. But the value it offers doesn’t imply anything about efficiency of use.
It only takes a few instructions in a Dockerfile to create a working Node.js application, for instance.
FROM node:6.9.5-alpine RUN mkdir -p /var/app1 WORKDIR /var/app COPY ./ /var/app RUN npm install --production ENV NODE_ENV=production NODE_PATH=./lib:. EXPOSE 3000 CMD ["node", "web/bin/www"]
But with this Dockerfile – pulled directly from a recent project I built – efficiency is not a word that comes to mind.
The problem comes down to the “RUN npm install –production” instruction.
While this may be functional, and it will install your node modules into your Docker image, it's going to destroy efficiency and productivity by requiring you to re-run the entire “npm install” every time you build your image.
And how often do you build your image? Every time you touch the Dockerfile, commit code changes to source control or update your dependencies.
If you’re not rebuilding your images frequently like this, then you’re out of sync with what’s going to be happening in the production image and that’s just dangerous.
But if you are rebuilding constantly like this, the time it takes for “npm install” to run will destroy your productivity!
… and it gets worse.
This is just the production image you’re dealing with so far.
And Then You Add A Development Image …
The production Dockerfile and its configuration is only a part of the process and problem.
When you're developing software inside of a Docker container, you also have to deal with development dependencies.
This list of additional dependencies may include build tools like Gulp or Grunt, code compilers or transpilers like Babel or Typescript, unit testing tools like Mocha / Chai or Jasmine, process management tools like Nodemon, and much much more.
Then there's the possibility of testing out new modules for new features during your development cycles which just adds more to the list.
All of these development dependencies have to be installed into your container. But none of them are going to be cached by Docker in the image that was created for production.
When it comes time to rebuild your development image to sit on top of the updated production image, guess what happens yet again?
You run “npm install” at least one more time, to get your development dependencies in place.
It only adds injury on top of insult on top of injury – like pouring salted lemon juice onto an open wound.
Is There A True Solution? Or Just A Workaround?
With the complexities of today’s applications and dependency trees sitting in side of Docker, it’s easy to understand why developers think they need to run “npm install” on a regular basis.
What’s not easy to understand is how the Docker experts handle this without suffering through this npm delay constantly.
How do they stay sane and productive when it can take 2… 5… 10 minutes (or more!) to run ‘npm install', and have it run multiple times per hour?
The answer is not “switch to yarn” as some would have you believe. Nor is a cup of coffee and Netflix the right way to go. Life is too short to spend it sitting idly, waiting for your package manager to finish its work.
The truth is, the Docker experts don't deal with this npm delay nearly as often as you think. In fact, they rarely run “npm install” unless they are building a new image stack from scratch or truly adding a new dependency to their application or development process.
Instead of looking for workarounds that might help cut some time from the install, the experts know the solutions to cut the npm delay by not running “npm install” unless they absolutely must.
A Solution For Everyone: Beginners And Experts Alike
Now, the expert's solution for creating the ultimate node modules cache inside of Docker is readily available to everyone.
Don’t wait an eternity for Docker to run ‘npm install', only to realize you need to change your Dockerfile and rebuild the image. Or that you have to run a new container instance and re-install all of your development dependencies… again.
Take advantage of the tools you already have in Docker. Build the ultimate Node.js module cache. Cut the npm delay from your Docker projects – in most cases, eliminating it entirely! – and regain your lost productivity!
A Solution For Developers, From A Developer
Dear developer, struggling with the npm delay in Docker,
My name is Derick Bailey.
I'm a developer, like you. I’m also a speaker and a published author. I've been writing code since 1988, have been a prolific blogger since 2004 and have been delivering world-class training to developers like yourself, through on-site classes, screencasts, ebooks and more.
I've spent a lifetime (more than 20 years) building software for leading financial organizations and airlines, the U.S. government, and more. I've created critical systems for large healthcare organizations, saving them millions of dollars a year. I've helped small-town, independently owned businesses improve their bottom line with simple software solutions and complex process automation.
I've also helped countless developers through sites like Stack Overflow, writing sample code for Microsoft's MSDN library, and by publishing dozens of leading open source projects, including the most widely used add-on for Backbone.js, called Marionette.js.
For some time now, I have been building software almost exclusively in Docker. It truly is one of the most important development tools I've ever seen.
I recently realized, however, that I was wasting hours of my life sitting and waiting, watching npm do it's thing inside Docker.
It became most apparent to me as I was delivering a live session in February of 2017. There were multiple occasions where I would have more than enough time to take a drink, comment on npm running, and still catch my breath while waiting for it to finish.
It's because of this experience that I dug in, dug deep and found the secrets to eliminating the ‘npm install' delay in most of my Docker work.
Now, I want to share the techniques that I found – hidden just under the surface of Docker's most commonly used tools – that have helped me the most.
Join me for this live session, and learn how you can cut the npm delay from your Docker projects.– Derick Bailey
– Creator of WatchMeCode.net
Kill the npm Delay in Docker
On March 27th, 2017, I delivered a live session of WatchMeCode where I will showed how to eliminate the `npm install` delay almost entirely from your Docker solutions.
In this 45 minute webinar recording, you’ll learn
- How most Dockerfiles are unintentionally set up to reinstall your node modules every time you build a Docker image or run a Docker container
- How one subtle change to your Dockerfile can mean the difference between countless minutes and split seconds of build time for your image
- How to stop Docker from re-running ‘npm install' when rebuilding your production image
- How to make a Docker container reuse a ‘node_modules' folder that already has your development dependencies installed – even after you rebuild your Docker images!
- How to use Docker volumes to speed up the few occasions where you must run ‘npm install'
- How to automate every every step of eliminating the ‘npm install' time so that you never have to think about it again
- … and more!
When the webinar is done, stay for the live Q&A with questions from yourself and the rest of the audience! This is your chance to ask me – a fellow Developer and Docker enthusiast – any question you want about writing code and developing apps in Docker containers.
Watch this intro to learn more about the webinar video!
Learn just how easy it is to cut your Docker build time when you eliminate the npm install delay from most of your Docker processes. Do it without having to install any new software. And do it with these three, simple techniques!
First, Fix Your Dockerfile
If you don’t fix your production Dockerfile, no amount of improvement with npm’s installation speed will make a difference. Running “npm install” multiple times per hour, only to re-install the same dependencies over and over and over again is not a good use of anyone’s time.
In this first technique, you’ll learn how to correct your Dockerfile so that you’re not forcing Docker to re-run “npm install” every time you build your production image.
Second, Cut The “npm install” Time In Half
Only after you’ve fixed your Dockerfile does it make sense to look at the performance problems with “npm install” inside of Docker.
While it can’t be reduced to zero when you truly need to install your dependencies, you can easily cut the time it takes in half for your development dependency list.
But cutting the time in half for this list of tools isn’t enough, on it’s own. You need to cut the frequency at which you install them from scratch, like you did for your production image.
Third, Reuse Your Node Modules Between Containers
By using a combination of tools and configuration settings found in Docker, with a little forethought and intelligence, you can create a caching layer for your development dependencies.
With this, you’ll install your dependency set once and reuse that set over and over again – even when you destroy your container instance and stand up a new one.
It's not a difficult technique, by any means.
And none of these techniques require additional software installation – only a better use of the tools that Docker has already provided.
Use The Tools You Already Have
With the techniques that you’ll learn in the live Caching Node.js Modules in Docker session, there are no special tools to install and no new software to configure and use. There are no changes to your project that require capital expenditure approval from any level of management, either.
You may not even need the blessing of your project’s lead developer to use these techniques, they're so simple!
You only need tools that you already have installed.
They came with Docker, with your operating system, or with the programming language you’re using (node.js is used in this video, but the principles apply to any language).
You’ll learn how to use existing tools and features to your advantage, and you’ll dramatically decrease the amount of time you spend idly waiting for npm install to complete.
Once you know the techniques above, you’ll wonder why you ever wasted hours of your life waiting for a Docker image build or installing development dependencies into a container.
These problems will be a thing of the past!
Use These Techniques Immediately
There's nothing worse than spending time and money on a solution that requires more time and money to implement.
With the techniques that I'll show you in the Caching Node.js Modules in Docker webinar, however, you'll be able to use the solutions you see as soon as the session is over.
No waiting for software to install or be configured. No delays with management and purchase orders.
The few changes you need for these techniques to benefit your project can be implemented in a matter of minutes. I'll show you exactly what needs to be done and how to do it.
And if that wasn't enough, I'll give you my working code and configuration from the webinar. Once you understand why you need to make these changes, you can simply copy & paste the needed configuration right into your Docker project.
With Caching Node.js Modules in Docker, you'll be the hero on your team, eliminating the npm delay instantly.
What Developers Say About Derick's Docker Training
“Thank you! Best docker resource I come across!”
“[the Docker cheatsheet] was useful, even though I've been
working with docker a while. Thanks for sharing that.”
– Oliver A
“Thank you so much for this cheatsheet –
it help in my start learning of docker.”
– Evgeny S.
“Yes. It's very good table to remember
the commands. Very useful! Cheers.”
– Arthur A.
“Thank you for the tips and advice. 🙂
Can't wait for more videos! keep it going! Thank you”
– Fabio F.
“Thanks Derick! I liked all the docker cheatsheets
with the graphic and short description it is perfect!”
– S. Thomas
Eliminate the npm Delay in Docker
The Caching Node.js Modules in Docker bundle includes:
- A recording of the live session from March 27
Including the live Q&A session at the end to address questions from the audience.
- A complete transcript of the video, after the live session
Including timestamps for video locations so you’ll never be lost between the video and the transcript. Instantly find the location you need!
- Full Code and configuration from the video
Including both the beginning and end state shown in the video. You’ll have everything you need to get started, in a single git repository!
You’ll also receive these additional bonus materials:
- A list of resources mentioned in the webinar and Q&A session
Find additional information and what you should be reviewing, next, to keep your Docker skills moving forward
- The set of 3 Docker Cheatsheets from WatchMeCode
Never again scroll through endless pages of online documentation. These three cheatsheets will show you the most common command-line options and Dockerfile instructions
- Opportunity to attend other live sessions at a discount
With the on-going series of live Docker webinars, you’ll have early access to the registration for each session at a discounted price
There is a complete course found in these materials, but you won't have to pay a full course' price to get this knowledge!
Your Price, Today: $49.00!
Buy the bundle now and learn how to create the ultimate Node.js modules cache for your Docker projects!
Kill The npm Delay – Guaranteed – Or Your Money Back
With my personal guarantee, you have zero risk in registering for this live session.
You’ll have 30 days in which to decide if the session was worth your time.
If you watch the session, apply the techniques found within it and the resources provided, and are convinced that you have not been able to eliminate any of the “npm install” delay from your Docker project, I’ll refund your purchase.
I am certain, however, that you will come out of this even with a dramatic increase in your productivity as you eliminate idle time from sitting and waiting for npm install to finish.