Improving meetups as the new organizer of the SF Node.js group

I couldn’t be happier to announce I am now the organizer of the 1750+ member-strong Node.js meetup group in San Francisco. I am extremely grateful to the previous organizer David Kordsmeier for giving me the reins. I pledge to maintain the same level of dedication and hard work that he has shown to the community. While I’m adjusting, David remains onboard as a co-organizer.

I started working with Node.js in 2010. Building realtime, bi-directional communication apps was a fascinating discovery and I was hooked. Since then I have traveled the world by working for Cloud9 IDE and now StrongLoop.

Traveling the world and attending all kinds of meetups has led me to a conclusion:

Meetups Can be Dramatically Improved

What’s wrong with meetups? Here are some common criticisms: The content is too technical or too watered-down; it doesn’t relate to any real-world application; it’s a product pitch; the content isn’t well-informed by experience; those who didn’t get into the limited venue space are upset; the location is inconsistent.

Some of these are circumstances that are out of our control as sponsors and audiences. But the heart of the meetup is the content, and that’s where the most improvement can be made.

Plain and simple: The audience needs to walk away with more hands-on coding experience. We want to start experimenting with the idea of hands-on workshops. In a workshop the audience codes their own applications, experiment with tooling, and walk away with code they can continue working on at home.

We will continue talking about this idea and how it can be accomplished at the meetup and in upcoming blogs. Stay tuned.

What Already Works

There is a lot to be grateful for in the Node community. Hundreds of thousands of us developers not only know JavaScript, but we actively contribute back to OSS projects. We willingly offer our time to help our fellow developers. We want to keep encouraging this spirited investment we have in each other’s success, and to keep the community atmosphere surrounding Node lighthearted and fun.

What has happened more and more recently is the grassroots Node developers have been knocking on the doors of their managers and introducing Node into their backend stack. This has reached bigger companies and enterprises such as Walmart, LinkedIn and Airbnb. Those companies put a high burden on their servers and their requirements have caused them to adapt in new and interesting ways, creating tooling along the way.

My focus as organizer of the group is to put more energy into bringing developers from enterprise in to talk about how Node works for them (or doesn’t) and what lessons they learned along the way. Indeed tonight is the first talk that fits this mold: Spike Brehm from Airbnb will be talking about Rendr and how it serves as a full-stack solution to their mobile needs.

Moving forward

Ben Noordhuis and Bert Belder are core contributors to libuv and are also co-founders of StrongLoop. We have a number of developers on the (growing) StrongLoop team that have years of experience with Node.js. So as we continue to create Node.js tooling, and provide support and consulting services to companies, we want to leverage our connections to the enterprise and community, and bring in great speakers for everyone to learn from.

If you have any questions or suggestions about how this can be improved, please let us know in the comments. Cheers!

What’s New in Node.js and libuv – April 25, 2013

Welcome to this week’s wrap up of the week in Node and libuv covering April 18-24. The purpose of this blog is to recap a subset of the non-documentation related commits to Node.js, plus give a little color and commentary to the ongoing development of Node.

Node v0.10.5 (Stable) is out!

An update to v0.10.5 dropped earlier this week with bug fixes and upgrades. Download v0.10.5 and read the official release notes on nodejs.org.

src: fix potential memory leak on early return

One omission to the release notes was a patch by Brian White (mscdex) that fixed a memory leak in crypto.Sign#final()’s error path.

Node v0.11.1 (Unstable) is out!

An update to v0.11.1 was released on April 19th with bug fixes and upgrades. Download v0.11.1 and read the official release notes on nodejs.org.

Please note that this version upgraded V8 to 3.18.0 that unfortunately broke support for 64-bit Windows systems. A fix is expected in the next release.

This week’s Node master branch highlights:

build: fix ARM build after v8 upgrade

Ben Noordhuis (bnoordhuis) fixed the ARM build after V8 was upgraded, so if you are a Raspberry Pi user using Node to build embedded apps, rejoice!

path: add path.isAbsolute(path)

Ryan Doenges (hackedy) added an isAbsolute() function to the the path module which means that now an absolute path will always open in the same location regardless of the current working directory. This addition fixed bug report #5299.

events: add EventEmitter.defaultMaxListeners

Ben added EventEmitter.defaultMaxListeners, which is a global override of the default maxListeners value.  It’s an old patch that had fallen through the cracks until the feature request was brought up again in issue #3014.

cluster: clean up lib/cluster.js

Ben rewrote the cluster module to clean up and DRY the cluster source code. In the process a few bugs were resolved. Here’s Ben’s commentary from the commit log:

  • “Short-lived handles in long-lived worker processes were never reclaimed, resulting in resource leaks.
  • Handles in the master process are now closed when the last worker that holds a reference to them quits. Previously, they were only closed at cluster shutdown.
  • The cluster object no longer exposes functions/properties that are only valid in the ‘other’ process, e.g. cluster.fork() is no longer exported in worker processes.

So much goodness and still manages to reduce the line count from 590 to 320.”

Last but not least, you can now download Node.js nightly builds for all supported platforms!

Actually, this has been possible for a while, but it’s now officially advertised on the Node downloads page under “Nightly builds”

You can view the complete Node master commit history on GitHub.

This week’s libuv v0.10 branch highlights:

linux: don’t use fopen() in uv_resident_set_memory()

Ben made the output of uv_resident_set_memory(), process.memoryUsage().rss in Node.js a little more reliable. uv_resident_set_memory() used to use fopen("/proc/self/stat") to retrieve the number of RSS pages. However, RSS is a reflection of the number of pages that a process has mapped. glibc implements fopen() in terms of mmap() which means that trying to read the number of mapped pages changes it, therefore the switch to open().

You can view the complete libuv v0.10 commit history on GitHub.

This week’s libuv master branch highlights:

unix: remove src/unix/cygwin.c

Ben removed Cygwin support in libuv. We are suspecting that nothing of value was lost here as it has been  broken for the better part of a year and not a single bug was filed.  This fact has been taken as a clear indication that no one actually uses it. If this isn’t the case, sound off!

unix, windows: add uv_has_ref() function

Saúl Ibarra Corretgé (saghul) added the uv_has_ref() function to libuv. This is a convenience function for checking if a handle has been unref’d with uv_unref().

 windows: deal with the fact that GetTickCount might lag

Bert Belder (piscisaureus) landed what Ben is describing as “a deeply evil patch” that “fixes” early timer expiry by changing the loop time. GetQueuedCompletionStatus(Ex) is used to sleep a thread until the next timer expires. (This assumes that no other events have occurred before then.) The rub is that after waking up from sleep GetTickCount() might return a value that reflects no time has passed. This can occur when gqcs sleeps for a period of time shorter than the GetTickCount clock resolution. The patch now forces accounting for the amount waited by gqcs.

 What’s the upside? According to Bert:

  • “Excessive loop iterations are avoided
  • Small timeouts are fired more precisely
  • The loop-stop test that used to be flaky on Windows now passes consistently.”

Evil indeed! 🙂

You can view the complete libuv master commit history on GitHub.

This week’s blogs, tutorials, how-to’s and news

What’s New in Node.js and libuv – April 18, 2013

Welcome to this week’s wrap up of the week in Node and libuv covering April 11-17. The purpose of this blog is to recap a subset of the non-documentation related commits to Node.js, plus give a little color and commentary to the ongoing development of Node.

Node v0.10.4 (Stable) is out!

An update to v0.10.4 dropped earlier this week with bug fixes and upgrades. Download v0.10.4 and read the official release notes on nodejs.org.

This week’s v0.10 branch highlights:

src: don’t SetInternalField() in ObjectWrap dtor

Ben Noordhuis (bnoordhuis) fixed a bug that caused some native add-ons to segfault in debug builds (and possibly release builds). Specifically, the fix involved calling SetPointerInInternalField(0, NULL) rather than SetInternalField(0, Undefined()).

cluster: fix O(n*m) scan of cmd string
child_process: fix O(n*m) scan of cmd string

Ben fixed a bug in cluster and child_process that caused massive slowdowns when sending large messages of a certain type between processes. Specifically, do not scan the whole string for a “NODE_” substring, just check that the string starts with the expected prefix. 

stream: Fix unshift() race conditions

Isaac Z Schlueter (isaacs) fixed a bug that addressed several race conditions in the way unshift() works in streams2. The behavior is now:

  • stream.unshift() does not set state.reading = false
  • stream.unshift() is allowed up until the ‘end’ event
  • Unshifting onto a EOF-encountered and zero-length (but not yet end-emitted) stream will defer the ‘end’ event until the new data is consumed
  • Pushing onto an EOF-encountered stream is now an error

In a nutshell according to Isaac, “So, if you read(), you have that single tick to safely unshift() data back into the stream, even if the NULL chunk was pushed, and the length was 0.” 

os: fix unlikely buffer overflow in os.type()
os: unbreak Windows build

Ben fixed two OS module bugs that in theory, are exploitable buffer overflows (but not in practice, unless you’ve already fully owned the machine.) The first patch fixed a buffer overflow that happens if strlen(info.sysname) was greater than 255. The second one fixed the Windows build, as it doesn’t support MAXHOSTNAMELEN which was introduced in the this commit. 

handle_wrap: fix NULL pointer reference

Ben fixed a NULL pointer dereference bug (which was back-ported to v0.8) that might go some way towards explaining why child processes sometimes go AWOL, something several people have complained about, but was never reproducible.  It’s not just child processes though, but anything backed by a libuv handle.  The test case, for instance, manipulates process.stdout to trigger a segfault.

You can view the complete Node v0.10 commit history on GitHub.

This week’s master branch highlights:

url: Escape all unwise characters
url: ~ is not actually an unwise char

Isaac fixed a url.parse() bug in 17a379e and 881ef7c where Node wasn’t escaping some ‘unwise’ characters (‘unwise’ in the RFC 2396 sense.) URL escaping should now be identical to how Chrome does it.

http: escape unsafe characters in request path

Ben fixed a bug in which unsafe characters in http.request({path: '...' }) weren’t escaped. The regression test Ben wrote for this bug was what uncovered the bugs that Isaac fixed above. 

Splitting up lib/http.js into smaller files

Timothy J Fontaine (tjfontaine) split up lib/http.js into several smaller files. Potentially not that exciting for the average users, but for developers at least, the http module is no longer the “impenetrable 2,000+ line monstrosity” that it used to be.

You can view the complete Node master commit history on GitHub.

This week’s libuv v0.10 branch highlights:

unix: dtrace probes for tick-start and tick-stop

Timothy added the first DTrace probes to libuv, yay!

windows: fix memory leak in fs__sendfile
windows: remove double initialization in uv_tty_init

Shannen Saez (shancat) fixed two Windows-only bugs: a memory leak in uv_fs_sendfile() and a uv_tty_init() double initialization bug.

windows: refactor uv_cpu_info

Bert Belder (piscisaureus) fixed a number of uv_cpu_info() bugs. The most important one being a free() of an uninitialized pointer in the error path. Here’s the complete fix:

  • Don’t free an uninitialized pointer when allocating memory for cpu_infos fails.
  • Don’t return a bogus error value when NtQuerySystemInformation fails. The function returns an NTSTATUS code instead of setting the last error.
  • Don’t clobber out parameters when an error happens.

build: add support for Visual Studio 2012

Nicholas Vavilov (seishun) added support for Microsoft’s Visual Studio 2012, closing out issue #722.

windows: make timers handle large timeouts 

Miroslav Bajtos (bajtos) – the latest engineer to join StrongLoop, fixed a Windows-only bug in the handling of timers with really large timeouts. The fix was based off Ben’s previous bug fix from back in March that addressed issue #5101. This commit fixed two closely related integer overflow bugs:

  • Timers with a timeout > INT_MAX causing uv_next_timeout() to return a negative value
  • Timers with very large timeouts (close or equal to ULLONG_MAX) running on the next tick

In both cases, the values were “clamped” to prevent the overflow from happening.

You can view the complete libuv v0.10 commit history on GitHub.

This week’s libuv master branch highlights:

darwin: look up file path with F_GETPATH

Ben fixed the uv_fs_event_*() functions to report the file name when watching a file for changes on OS X.  That is, touch watched_file now reports the file name and mv watched_file renamed_file reports the new file name.

You can view the complete libuv master commit history on GitHub. 

This week’s blogs, tutorials, how-to’s and news:

What’s New in Node.js and libuv – April 11, 2013

Welcome to this week’s wrap up of the week in Node and libuv covering April 4-10. The purpose of this blog is to recap a subset of the non-documentation related commits to Node.js, plus give a little color and commentary to the ongoing development of Node.

Node.js v0.8.23 is out!

A maintenance release to v.0.8 dropped earlier this week with bug fixes, backports plus an upgrade to npm. Download v0.8.23 and read the official release notes on nodejs.org.

Node.js v0.10 branch highlights

stream: call write callback before finish event

A patch was added to fix a bug that was causing file sizes to register as “0”  and the lastModifiedDate value to be “null” in node-formidable, a popular module for parsing data, especially file uploads. The patch also got to the underlying issue of #5215 which Isaac described this way: “The root problem is that finish is emitted in onwrite when we call finishMaybe. Then, later, we call afterWrite which actually calls the write callback.”

crypto: Diffie-Hellman secret should be left padded

Fedor fixed issue #5239 by adding left-padding to keys that are smaller than the input buffer so that keys can now validate consistently.

tls: Re-enable check of Common Names-ID in certificate verification

Tobias re-enabled the checking of Common Names when verifying a TLS certificate. This means that when a certificate has no DNS-ID, SRV-ID, URI-ID or any application-specific identifier, it will check CN-ID.

Excessive Memory usage in JSON.parse( )

A patch from upstream V8 was added that fixes excessive memory usage in JSON.parse().

Node.js Master branch highlights

Ben upgraded the master branch to V8’s 3.17.16 version.

This week’s libuv highlights

unix: include uv.h in src/version.c

A fix was implemented to correct the shared object build which was previously missing symbols due to a symbol visibility mix-up. This in turn was causing the Fedora RPM build to fail.

You can view the complete Node master, v0.10, and libuv commit history on GitHub.

Node.js blogs, tutorials, how-to’s and news:

What’s New in StrongNode Beta 3 – Private Repositories

Last month, we introduced slc, a command line tool that ships with the StrongNode distribution for building and managing applications. It’s a Swiss-Army knife tool that provides commands for scaffolding, debugging, testing and documenting Node.js source code. With today’s release of StrongNode Beta 3, we are introducing new functionality to the command-line tool that lets you install packages from the StrongLoop Node repository, the community repo and/or a private repo. That’s right StrongNode now has support for private repos!

What does support private npm servers mean?

Just like it says on the tin. We’ve added functionality to slc  so that it is now possible to download packages from multiple npm servers.

Why did we build support for multiple npm servers?

Even though at the core of the StrongNode distribution are a set of vetted and supported modules, we wanted to make it possible for users to still take advantage of the modules offered by the community and those which might be maintained in private repositories. A perfect example where this might be valuable would be a company that is building a Node application and has decided to build its own modules. They’ve also decided to maintain them in a private repo. While this company doesn’t necessarily want to share these modules with the public, it still wants the ability to pull in modules from the community npm and the StrongNode repo.

How does support for multiple npm servers work?

Behind the scenes, to make npm query both the StrongLoop and the “official” npm registry, we’ve extended the way npm parses the registry configuration option. Previously, the registry option was simply supposed to be an url. With our changes it is now also possible to make the options into a hash table, which allows you to specify per-repository configurations. That means you can add something like the following to your .npmrc file:

The priority option may need some explanation. When npm is installing a package, it’ll start querying the registry with the highest priority. If a requested package isn’t found in the preferred registry, it’ll search other registries as well. When publishing a module, it’ll simply pick the registry with the highest priority.

It is also possible to override the priority used for a specific purpose with the install-priority and publish-priority options. This lets you publish to a particular repository while preferring another when installing packages.

By default, the slc command-line tool queries the StrongLoop repository first and then the community npm. By default, the slc command-line tool queries the StrongLoop repository first and then the community npm. You’ll want to note that the “install” and “list” commands will warn you if a module is not supported in the StrongNode distro.

There are some rough edges left that we’ll work out in the near future. For example, it is currently not possible to manually override the repository to operate against. That means that authenticating yourself with a particular repository currently isn’t easy. Also notice that, although we intend to make this feature available in the official npm client, it hasn’t gotten there yet, and is currently available only in the slnode client. We’ve modified the npm client to support multiple npm repos, but that’s only half the equation.  In a follow on blog post we’ll go over how to set up your own private repository. Let us know what you think!