Digging into the requirements

Creating a full-blown solution like this using serverless only can seem like a daunting task at first. However, we knew we could deal with most of the requirements by using the following built-in functionality of restdb.io:

• Create collections (and API) for holding email templates and for recipients
• Markdown field type (for editing email templates)
• Unique field constraints (for the email field of recipients)
• Import CSV/Excel
• Codehook background jobs (for sending the emails robustly independent of the web client)
• Mail API and unsubscribe functionality
• Pages (for the dynamic HTML/JS user interface)
• Realtime (for updating campaign stats live)

HTML User Interface

The HTML user interface was created using restdb.io Pages using both Javascript AJAX REST calls and server side database queries. It is actually quite simple and consists mainly of a link to restdb.io's data import, a simple dropdown for picking and previewing the template and a status of active campaigns and of course a button to run the campaign (not as cool as MailChimp's sweating monkey pushing the "Send" button, though).

Clicking on the "Campaigns" tab shows past and running campaigns along with progress information and open percentage. One cool thing not possible to see here, is that the numbers are updated live. This is possible because you can listen to database REST events (in this case a PUT on the campaign job record).

We used the following Javascript/HTML libraries:

• jQuery (to listen to events and manipulate the DOM)
• Bootstrap (quick'n easy HTML UI)
• Bootbox.js (alerts and dialogs - here used to confirm that the test email was sent)
• bootstrap-select (for selecting the email template)
• Lodash (utility library based on Underscore.js)
• Marked (Markdown parser/compiler used for Markdown preview)
• Superagent (for simple and elegant AJAX requests - used for all REST calls)
• MobX (for simple state management)
• restdb.io Javascript API (for the realtime updates)

To view (or modify) the source code, just install the template into a new database, activate "Developer Mode" and click "code editor" just below the "Dashboard" menu item (as shown in the image below).

The Javascript source can be found in the index.js Page (shown below).

NoSQL Database Setup

We created basically just two collections (tables) to hold our data:

• recipients (with field "email")
• emails (with most important field "body" - containing the email template in Markdown syntax)

Other important collections are the built-in system_jobs and the email_outbound collections (more about these later).

Creating and editing emails

This functionality is supported directly with restdb.io. For any field on a collection which is of type text and "flavour" Markdown, will give you some editing help when creating a record in the database backend.

Background Hooks for sending emails

One of the crucial parts of restdb.io which makes it possible to create an application like this, is Background Hooks.

We needed a background job for this application for a few important reasons:

• POSTing thousands of records from the client would most likely fail
• Sending emails can fail. We need to know which has been sent.
• There are limits to how many emails it is possible to send in one batch
• Serverless code are usually time-limited (to avoid them from hanging the servers)

A Background Hook is technically a record in a predefined system collection called system_jobs with a few predefined fields: crontab (a way to schedule the job - similar to *nix Crontab), active - a boolean indicating if the job is active and script - the Javascript code to be run.

For this solution, we added a few fields to the system_jobs collection: jobtype, status, newsletter. Jobtype was added to separate this type of job from (potential) others, status is the sending status, newsletter is a lookup relation to the newsletter to be sent.

Putting it all together

To give you a better understanding of the application, we'll describe the basic flow of using it:

• Add recipients (upload files or use REST API)
• Write an email (a record in the email collection)
• Test the email (default is the database owner). This uses client code to create HTML from the Markdown and then sends a POST (using Ajax) to the /mail endpoint of the database.
• When clicking the Run! button, the application Ajax POSTs a new system_job record containing the job script and links the email to it (with a lookup relation). The job will run every minute and tries to send as many emails as possible before being "killed" by the timeout.
• restdb.io Mail service automatically updates the email_outbound collection when emails are sent
• The stats are updated live using the realtime API. We use a special collection countdata with codehooks and aggregation to get the tracking-status of sent and read emails.

The figure below shows a schematic overview of how the background job works.

Conclusion

Creating a fully functional serverless application for sending emails is certainly doable as we've shown. It took us a few iterations for us to get the solution from the idea stage and into something we could actually use. The difficult part here was not writing the actual code, but to find out which code to write (somehow it's always like that!). The resulting code is not many lines in total, around 550 lines of Javascript in addition to a little HTML.

Next steps for you could be to test out the template and play around with the code (it's free and open source).

If you found this article interesting, please share with your fellow developers!

Links

• Mail API
• Codehook API
• REST API
• Email Campaign template