Digging into the requirements
Creating a full-blown solution like this using serverless only can seem like a daunting task at first. However, we knew we could deal with most of the requirements by using the following built-in functionality of restdb.io:
- Create collections (and API) for holding email templates and for recipients
- Markdown field type (for editing email templates)
- Unique field constraints (for the email field of recipients)
- Import CSV/Excel
- Codehook background jobs (for sending the emails robustly independent of the web client)
- Mail API and unsubscribe functionality
- Pages (for the dynamic HTML/JS user interface)
- Realtime (for updating campaign stats live)
HTML User Interface
Clicking on the "Campaigns" tab shows past and running campaigns along with progress information and open percentage. One cool thing not possible to see here, is that the numbers are updated live. This is possible because you can listen to database REST events (in this case a PUT on the campaign job record).
- jQuery (to listen to events and manipulate the DOM)
- Bootstrap (quick'n easy HTML UI)
- Bootbox.js (alerts and dialogs - here used to confirm that the test email was sent)
- bootstrap-select (for selecting the email template)
- Lodash (utility library based on Underscore.js)
- Marked (Markdown parser/compiler used for Markdown preview)
- Superagent (for simple and elegant AJAX requests - used for all REST calls)
- MobX (for simple state management)
To view (or modify) the source code, just install the template into a new database, activate "Developer Mode" and click "code editor" just below the "Dashboard" menu item (as shown in the image below).
index.js Page (shown below).
NoSQL Database Setup
We created basically just two collections (tables) to hold our data:
- recipients (with field "email")
- emails (with most important field "body" - containing the email template in Markdown syntax)
Other important collections are the built-in
system_jobs and the
email_outbound collections (more about these later).
Creating and editing emails
This functionality is supported directly with restdb.io. For any field on a collection which is of type text and "flavour" Markdown, will give you some editing help when creating a record in the database backend.
Background Hooks for sending emails
One of the crucial parts of restdb.io which makes it possible to create an application like this, is Background Hooks.
We needed a background job for this application for a few important reasons:
- POSTing thousands of records from the client would most likely fail
- Sending emails can fail. We need to know which has been sent.
- There are limits to how many emails it is possible to send in one batch
- Serverless code are usually time-limited (to avoid them from hanging the servers)
A Background Hook is technically a record in a predefined system collection called
For this solution, we added a few fields to the
system_jobs collection: jobtype, status, newsletter. Jobtype was added to separate this type of job from (potential) others, status is the sending status, newsletter is a lookup relation to the newsletter to be sent.
Putting it all together
To give you a better understanding of the application, we'll describe the basic flow of using it:
- Add recipients (upload files or use REST API)
- Write an email (a record in the email collection)
- Test the email (default is the database owner). This uses client code to create HTML from the Markdown and then sends a POST (using Ajax) to the
- When clicking the Run! button, the application Ajax POSTs a new
system_jobrecord containing the job script and links the email to it (with a lookup relation). The job will run every minute and tries to send as many emails as possible before being "killed" by the timeout.
- restdb.io Mail service automatically updates the
email_outboundcollection when emails are sent
- The stats are updated live using the realtime API. We use a special collection
countdatawith codehooks and aggregation to get the tracking-status of sent and read emails.
The figure below shows a schematic overview of how the background job works.
Next steps for you could be to test out the template and play around with the code (it's free and open source).
If you found this article interesting, please share with your fellow developers!