gander meteor.png

When we started prototyping Skimbox using Meteor's open source web app platform, we weren't sure the Shiny New Thing with its CSS/jQuery/node.js/Mongodb stack and symmetric real-time data model had legs capable of carrying our email app into the real world. Our relationship in the two months since hasn't always been angels and butterflies, but it has been fruitful--enough so that we're (for now) planning on keeping it for Gander's live version (still in closed beta, but you can sign up for the waitlist here). 

The following list contains information on Meteor's features, third-party tools, Q&As, things to watch out for, events, and best-practices. They've proven useful to myself and our team; hopefully they are to you as well. If you have further questions, though, don't hesitate to ask them in the comments, on twitter, or on Stack Overflow.  

General Links

Meteor Main Page

  • This includes links to the documentation, lots of other resources, Stack Exchange tags, etc.
  • Worth following on twitter http://twitter.com/meteorjs

Matt DeBergalis talk at Realtime 2012

  • Matt describes an overall view of Meteor and the underlying DDP protocol.

David Greenspan introduces Spark 

  • Demo and brief explanation of reactivity, Live HTML and templates

Debergalis Oct 2012

  • Introduction to Meteor, including authentication and reconnection demos

Tom Coleman's Github Page

https://github.com/oortcloud

  • Includes some potentially useful tools for packaging Meteor, either to include new packages, or to deploy to Heroku instead of meteor.com

EventedMind training videos

  • This guy has produced an awesome set of training videos to learn Meteor. This will be very valuable for newbies to Meteor. The one on DDP is especially useful. Great stuff.

http://andrewscala.com/meteor/

  • Nice simple tutorial for Meteor. 

How to debug Meteor server-side. Client side debugging is typically done using the Chrome Developer Tools. See also https://github.com/meteor/meteor/pull/412

Test Driven Development in Meteor pretty comprehensive SO question about using TDD in Meteor

How Does Reactivity Work Behind the Scenes  another pretty comprehensive SO answer explaining how Meteor's reactivity works.

Some Things to be Aware Of

Minimongo Restrictions

Since data is replicated on the client in 'minimongo', an in-memory copy of the data,  there are some restrictions in the client:
  • The MongoDB _id default cannot be used because it is a BSON object for efficiency. Instead a straight text _id must be used like an RFC4122 UUID
  • Indexes are not available client side. Since minimongo is entirely in memory, this should still be fast, but don't send down 100,000 items and expect fast lookups.
  • Since entire collections are copied, there is no point is sorting server-side. For display purposes, sorting has to be done on the client.
  • Minimongo currently does not allow sorting on subkeys (ie. obj.headers.Subject) (This restriction was removed in Meteor 0.5.3 and later).
  • Minimongo supports only String, Number, Boolean, Array and Object types (ref). Binary dates must be converted to strings.

Meteor uses Fibers, not Node.js Async

Meteor uses fibers, which is debatable. In some ways, it makes it easier to program server-side code. OTOH, this is inconsistent with the majority of node.js Node Packaged Modules. It isn't clear how to mix and match these two different synchronization approaches in a single Meteor app. It also isn't clearly documented how to use fibers in a custom package to avoid blocking and other messiness. Further reading: TechCrunch and the sparse Meteor docs, and this performance related SO question (see also the odd comment from Tom Wijsman at the bottom), Quora.

There's a pretty helpful gist that demonstrates a number of different techniques for handling async behavior and node integration with Meteor.

Mongo Access is Serialized and Synchronous

When looking at the packages/mongo-livedata package, it appears that Meteor puts all Mongo in a fiber, which serializes it. It also uses Mongo's safe mode which is synchronous. This leads to a simpler, more reliable implementation. However, this model will introduce scaling problems under higher loads. Mongo supports asynchronous writes, with callbacks, and as well multiple concurrent calls. It is typically used that way in node.js deployments. The Meteor team will likely have to do some non-trivial rewriting of their driver for higher performance.

Publishing User Defined Fields is Poorly Documented

If you add fields to the user's collection, retrieving them later in the app is poorly documented. On the server, follow the documentation:

Meteor.publish("userData", function () {
 return Meteor.users.find({_id: this.userId}, {fields: {'other': 1, 'things': 1}}); 
});

On the client, subscribe to this collection:

Meteor.subscribe("userData");

Now the new fields will appear in Meteor.users, as in:

Meteor.users.findOne()['other'];
Meteor.users.findOne()['things']; 

So even though the subscription is named differently, the data still shows up in the users collection. If the client does not subscribe to this new collection, the additional fields will not be available. (I had to file a bug report to understand this).

Reactivity is not Guaranteed

We have found two places where Meteor's reactive context does not work as advertised.

  • The subscription callback for completeness. This callback will be invoked before the collection is completely loaded on the client. According to a recent SO question, DeBergalis contradicted the documentation stating "subscribe() takes a callback that will run when the initial [emphasis mine] set of documents are on the client." It seems that the publisher will call OnComplete, triggering the client-side callback, prior to the low level shipping of the full collection. Bottom line: it is non-deterministic to know if the collection is fully populated. UseMeteor.call to invoke something guaranteed, synchronously. 
  • Autorun will not execute when some of the dependent Session variables or collections change.

While we don't know for certain, it seems like reactivity breaks down when there are multiple dependencies. If one reaction changes something that should set off a second reaction, the second reaction does not fire consistently.

As a workaround for the subscription callback, it is always possible to use a separate Meteor.call to the server. The server method could return an aggregate object. When the client Meteor.call callback is invoked, the full results are present in the callback. So this is still asynchronous processing but the results are definitely on the client when the callback is invoked.

Synchronous Usage of Meteor.call Does Not Return Results

When using Meteor.call in the synchronous form, this is just a stub and there will be no return value from the server. Instead you have to set up an asynchronous call to capture the correct result.

Spurious Error Message Due to Mal-Formed HTML in a Template

It took a couple of days to track down a bug in Beaver. The browser's page would crash hard, where even a hard reload was not working consistently. In the browser console, there was a very long stack trace starting with:

Exception from Meteor.flush: Error: An invalid or illegal character was specified, such as in an XML name. at Function.Spark.Patcher.copyAttributes

The problem was due to missing a closing '>' in the HTML of the template. Meteor crashes hard if the HTML is mal-formed in a template. Unfortunately the HTML checking tools will not work on templates, especially partials so this must be done manually and carefully.

<td><input type="checkbox" id="show_subcategories"

 {{#if preferences.show_subcategories}}

 checked="true"

 {{/if}}

 >        <---- This was missing

</td>

No Deny without Allow

Meteor allows deny and allow methods in order to implement security on Insert, Update and Delete operations. While it is possible to have allow methods without deny, it is not possible to have deny without allow. If a given collection has a single deny method, it must have at least one allow method. This is buried in the docs as:

"If you never set up any allow rules on a collection then all client writes to the collection will be denied, and it will only be possible to write to the collection from server-side code."

Open Questions

None at this time.

Closed Questions

What storage limits are there to databases stored on meteor.com?

  • Hosting on meteor.com does not currently have any data caps, but it also has no SLA or guarantees =). If someone abuses the hosting service with unreasonable data or bandwidth usage, we may very well add data caps, but we haven't had a problem yet.

How often does the client do a full refresh? SO claims every 10 ten seconds. T/F?

  • True, but misleading. 10 seconds is the time it takes the server to notice documents added directly to the mongo database by an external process. When clients write to the database through Meteor, other connected clients see it immediately, not 10 seconds later.

Is packaged Mongo 32 or 64 bit?

  • MacOS and Linux x86_64 ship with 64 bit mongo. You'll only get a 32 bit mongo if you are running on a 32 bit linux machine. meteor.com is running 64 bit mongo (hosted by MongoHQ).

Does Meteor play nice with Mongo's Auth package, or does it require no Auth?

  • Meteor's auth works at a level above the database. It does not integrate directly with Mongo's auth.

What ports are required to be opened on a Meteor server besides standard 80/443?

  • Meteor listens for incoming HTTP requests on whatever port you tell it (3000 by default in local development, 80 by default when run via the meteor bundle command). The meteor bundle does not handle HTTPS, that is a feature of the meteor.com hosting. The meteor development mode runner takes an addition internal port and runs mongod which takes 2 more ports. But these ports are not used externally, and should not be open.

According to SO, "$ meteor mongo -U" is valid for only one minute. What's a more reliable way to determine this persistently?

  • That's correct. The one minute thing is a feature, designed to limit the risk of your database being attacked. If you want a permanent URL, though, you can work around this. Your deployed server is passed a URL to a mongo endpoint as the "MONGO_URL" environment variable. You can print 'process.env.MONGO_URL' in server code to see it. Be careful though, because we don't expose a good way to change your database password short of deleting the app and re-deploying.

Can regular Mongo monitoring services be used against Meteor?

  • In practice, it doesn't matter. Any serious application should be using a stable Mongo database hosted elsewhere, like MongoHQ. When deploying to meteor.com, Meteor requires hardcoding credentials in a local copy of Meteor for accessing the MongoHQ instance. Long term, it is preferable to run a Meteor app elsewhere like AWS or Heroku, rather than meteor.com.

Related Projects

Meteor either encapsulates or plays nice with

  • Handlebars (HTML templating)
  • jQuery client JavaScript library for eventing
  • node.js server Javascript execution environment v0.8
  • bootstrap the scaffolding / column layout manager used by Mahogany

The partial list of third party packages for Meteor can be found at https://atmosphere.meteor.com/ (DW: doesn't the atmosphere cause meteors to burn up and vaporize?)

Alternatives to Meteor

See Also

Meteor Google+ which includes links to a bunch more talks.

Comment