Why I needed Noderunner

Sat, 03 Jan 2015 18:46:34 GMT

I have lots of little scripts scattered across my servers whose only job is to keep to things in sync.

An example

For example, I edit Scripting News in Fargo, but serve it on S3.

S3 is necessary for Scripting, because so many people subscribe to the RSS feed, it's the most practical way to scale it. Let S3 handle the traffic. They have to do it for everyone else too.

So I need a little bit of glue that's watching a folder and when something new shows up copies it to S3.

20 years of glue

I've been doing this for 20 years! So there's lots of little crap floating around. And I'm always creating new stuff that needs to be glued together. I didn't want to deploy the new stuff on various Frontier servers I still run (have to or else the content goes away), I want to do it all in Node.js, and run on Unix, from this point on. So Noderunner is where I write the little cheesy bits of glue I need to keep things going as I develop more stuff to glue together next month.

Another example

I have a NYT river that flows items into the @NYT twitter account. That connection done by a little bridging script that runs once a minute.

Why Dropbox is cool for this

Keeping it all in the filesystem makes it easy to do an audit, and also means I have a wide variety of choices of tools to use to edit these scripts now and in the future. Hooking it all up via Dropbox means I can edit stuff no matter where I am. I could even edit server code on my iPhone because they have a Dropbox that runs there too.

Noderunner is duct tape

You live to write elegant software where everything flows naturally.

Unfortunately it doesn't always work out that way. Or history takes us in a different direction. For those situations you need places to organize your little hacks that glue things together. For that, a script organizer like Noderunner is really nice to have.

"It's even worse than it appears!"

Noderunner v0.57

Tue, 30 Dec 2014 17:39:43 GMT

Two major changes in this release of Noderunner, thanks to a suggestion by Marco Fabbri on the mail list.

folderPath

If you specify a noderunnerFolderPath environment variable, Noderunner will store its and and run scripts from that folder.

export folderPath=/home/bull/mydata/

If you don't specify it, the functionality is as before, the data and scripts are expected to be in the same folder as noderunner.js.

You may want to create the folder running the app and put prefs.json in the prefs folder. I needed to do this because I have Noderunner responding to HTTP requests on a port other than the default, port 80.

Line-endings

I changed the line-endings in noderunner.js and the sample scripts to be Unix style, a single \n character.

Re Marco's other comments

The "web" folder was part of a feature that was removed while Noderunner was in development. It became another snack, which should be the next one released. I felt it needed to be separate, keeping with the "do one thing well" philosophy. The project was drifting, and I needed to refocus it.

Re using eval, there are tradeoffs to all script-running approaches. I stuck with eval over the other options, because it offered the simplicity and power I was looking for. I want user scripts to be able to access the routines and data of Noderunner. I want to add more built-in verbs, and work on it as I worked on the verb set of Frontier.

I know that the vm package allows you to provide a sandbox for scripts to run in. But I don't want to have to think about this too much, Noderunner is important, but it's not everything. And maybe at some point its functionality will be absorbed into something designed on a grander scale.

Noderunner is designed to be a nice utility for an individual programmer, or a small team. You don't need to protect the environment any more than you would for other code you write. If you wanted to offer this as a service to random people, you would most definitely need to sandbox the scripts. But that's now how I'm expecting to use this.

The other items on Marco's list are great, you can tell that I'm new to working in this environment. I developed in Unix in the 1970s. My whole philosophy of development was formed by this experience. But I spent the 80s, 90s and 00s working in Mac and Windows. A lot of new Unix culture formed in that period, and I forgot stuff I used to know. Thank you for your patience!