This is a single archived entry from Stefan Tilkov’s blog. For more up-to-date content, check out my author page at INNOQ, which has more information about me and also contains a list of published talks, podcasts, and articles. Or you can check out the full archive.

Jonathan Weiss on Rails Patterns

Stefan Tilkov, Jun 10, 2008

These are my unedited notes from Jonathan Weiss's talk "Rails Patterns".

Knowledge gained in large-scale Rails projects
Typical issues: mdeia conversion, user data, lots of plugins, external dependencies, slow requests* LOVDbyLESS example
Using Stephen Bristol's LOVDbyLESS as an example
Pattern approach: context, problem, solution

Media conversion

Scenario: Users want to upload some type of data
Typical solution: do conversion inline (esp. with attachment_fu) or system call to ffmpeg
conversion happens within HTTP req/resp cycle (bad)
Problem: conversion takes time: rails process blocked, risk of connections dropped, user sees blank page, all resources consumed on app server; risk of both intentional as well as unintentional DOS attacks
Solution: process media files asynchronously
Pattern: Separate long-running and resource-hungry tasks
options: use of message bus, fork process; often best b/c easiest solution: poll a DB containing jobs [ed.: Don't want to copy all of Jonathan's presentation – sticking to context, problem, solution only now]

Handling user generated data

Scenario/Context: users upload lots of data; usual approach: storing data locally, or with NFS, or on S3
Problem: NFS not scalable, S3 upload too slow, background jobs need to read/write from S3 (S3 problems now reduced because of local approaches)
Solution: save locally (fast); schedule asynchronous S3 upload (e.g. via message bus)
Audience question: S3 in Europe now? Jonathan: yes, but AWS:S3 doesn't work with S3 Europe (attachment_fu w/ S3 backend doesn't work at the moment)

You want to extract contacts from user email accounts

Scenario/Context: You want to extract data, search for a plug-in, pick the first one
Problem: most plugins aren't well maintained, have poor quality, are hard to adjust.
Even popular plugins have problems: Backgroundrb, acts_as_ferret, restful_authentication
Solution: Do not blindly use plugins
Pattern: Choose your dependencies wisely
Often, adding the missing 10% to a plugin is harder than coding yourself
Re-use is overrated :-)

External Dependencies

Scenario/Context: Your applications have several external dependencies, in specific versions, installed by different means (package management, source compile, gem, plugin)
Problem: deployment becomes brittle
Solution: "vendor everything" (all libs and dependencies in /vendor)
Pattern: Freeze your dependencies

Your site allows people to upload and download files

Scenario/Context: limited number of mongrels (typical: 500,000 PIs per day/use 10-20 CPUs, 6-8 mongrels each)
Problem: Denial of Service Attacks very easy: simply start many down/uploads over a slow/throttled line
Solution: contaminate slow requests: serve static files through webserver, define several clusters for different tasks, redirect depending on URL
Pattern: Use the right tool for the job - consider e.g. using Merb, Pure CGI, C, Perl to handle uploads/downloads
new version of mod_rails/Passenger buffers uploads

Conclusion

Real life solutions are more complex than they seem
Don't be too fast with throwing foreign code at a problem
Look for patterns in other people's work

Another very good talk.