Technology Web

Developing Apps with Environment Variables

This post is mostly a reminder to myself about my setup to deal with environment variables when working on web applications. I develop in Ruby on OS X, to give you the gist of my setup.

It wasn’t until I switched from ASP.NET and C# to Ruby that I started using Heroku. Heroku is awesome, but has its opinions of how apps should be setup. It was a little bit of a mental shift coming from Microsoft technologies, but I love it now.

Importing & Exporting Config Variables – The Heroku CLI

Heroku wants you to store all credentials and other settings in environment variables. This allows you to modify credentials and such without affecting the code. It also makes sure that the code doesn’t have sensitive information in it when checked into git.

Heroku’s command line utility lets you easily list and update environment variables on Heroku. However, it’s not very easy to get those environment variables loaded when developing on your local machine. There’s a plugin for Heroku’s command line utility that lets you export environment variables to a “.env” file. I’ve created a modified version of the plugin that lets you specify separate files for each environment (e.g. development.env, staging.env, production.env). These files can be modified and pushed back to Heroku easily.

Loading Environment Variables – Using Foreman

Now that we can manage the environment variables on Heroku, how do we use them locally? Well, that’s what Foreman is for. It was created by David Dollar, along with the original Heroku CLI plugin that I tweaked. The beautiful thing is that Foreman uses the same “.env” files and lets you specify the filename to load.

A quick step back: Foreman lets you easily start up multiple processes required to run your web app. You can specify what processes to start in a Procfile, which is the basis for Heroku web apps.

Now, you can just run a command such as “foreman start” and it will load up the Procfile in the current directory. You can also specify your environment variables to load such as “foreman start -e development.env”. Your Procfile might simply have “web: rails s” in it, so Foreman just runs that command with environment variables automatically loaded from your file.

The latest piece of the puzzle I put together was to get the environment variables loaded when I wanted to use the Rails console. Foreman has a nice little hidden command which is “foreman run”. The latest version as of a day or two ago lets you use your “.env” file. All you have to do is run “foreman run -e development.env rails c” and you’re good to go!

Simplifying Your Development Environment – Use Shortcuts!

Last but not least, I created some shell functions that really easily let me run any command with my environment variables loaded. Additionally, I wrap my command in “bundle exec” where appropriate to make sure the proper gems are loaded when running my command. Here they are:

function dev() {
  if [[ -f Gemfile && [email protected] != bundle* ]]; then
    foreman run -e development.env bundle exec [email protected] ;
    foreman run -e development.env [email protected];
function prod() {
  if [[ -f Gemfile && [email protected] != bundle* ]]; then
    foreman run -e production.env bundle exec [email protected] ;
    foreman run -e production.env [email protected];
function stage() {
  if [[ -f Gemfile && [email protected] != bundle* ]]; then
    foreman run -e staging.env bundle exec [email protected] ;
    foreman run -e staging.env [email protected];

All you have to do now is prefix any command with “dev”, “prod”, or “stage” to run the command with those environment variables. For example, I’m constantly running “dev rails c” and everything works flawlessly!

Update: Using Environment Variables in Your Gemfile on Heroku

I am hosting some private gems via git and to access them Rails needs a username and password. By now, you know where the credentials are stored. However, Heroku does not load environment variables when building the slug. To enable this, you need to install a Heroku “labs” plugin by running this command:

heroku labs:enable user_env_compile -a myapp

Hopefully all my bases are now covered…..

Technology Web

Pre-populating Shopify Checkout Forms and Other Undocumented Functionality

We’ve been using Shopify at ThankLab and chose them because they’re a hosted solution and have a simple API. However, we quickly uncovered many limitations that required us to come up with workarounds. Some of the issues we had to resolve were a result of integrating Shopify into a third-party website with existing user accounts.

As we dug into documentation on the official Shopify wiki and blog, it became clear that many of the solutions they provide are their own hacks around limitations many customers have run into. This realization was double-edged: Shopify clearly encourages hacking on their platform, but many capabilities or “features” are not really baked into their core product. In the end (frustrations aside), the challenge of making Shopify work for us became an exercise in creativity.

Seamlessly integrating third-party user accounts with Shopify checkout

Our users login to our website and checkout on Shopify. Shopify has these 3 options at checkout:

Shopify checkout options

  • Guest checkout – requires the customer to enter their email address
  • Customer login only – requires email and password to login, however each customer needs to be manually invited via email to create passwords for their account

Goal: pre-fill email address on guest checkout page (and hide email field)

Shopify limitations: Checkout page has a unique URL (is not known ahead of time), does not accept query string parameters to pre-fill form fields, and only allows you to customize CSS.

Get the Shopify checkout URL ahead of time

I kinda lied there a little bit. The checkout page CAN be determined ahead of time, but because the URL has a random looking token in it, it isn’t immediately evident. Here’s an example of my cart on the Penny Arcade store:

Shopify checkout URL

As it turns out, the ID in the URL is your shop ID (429942) and is always the same for all customers. The 32-character ID (c813f407436dc8cdda0c50d58e6d5e96) is your cart token, which is stored in a cookie called “cart”. To get the checkout URL for each customer, you’ll have to use JavaScript:

  1. Get your shop ID from your checkout page URL (simply click “checkout” in your shop)
  2. Create a JavaScript function that can read the “cart” cookie (here’s some code you can use)
  3. Create a JavaScript function that combines everything into a proper URL and do something with it

After combining these steps, you might have some JavaScript code that looks something like this:

var shop_id = "429942";
var checkout_url = "" + shop_id + "/" + readCookie("cart");

We will use this URL to add some parameters to pre-fill the checkout form. Download my example script (shopify_hacks.js) from Github.

Pre-filling the Shopify checkout form

We are now able to determine the Shopify checkout URL ahead of time, but need to pre-fill the form. If you’re familiar with Ruby on Rails or web development frameworks in general, you know that you can typically take the names of form fields and add them to the URL as query string values. Those values are then pre-populated in forms that have fields matching those parameters.

In the case of the checkout form, if you view the HTML source of the page in your browser and find the “input” tags, you’ll find the field for the email address on guest checkout. The name is “order[email]”. Now, when you add that name as a query string value, the checkout URL looks like this:

Shopify checkout URL with email parameter

Unfortunately this didn’t work. The email address is still blank. However, if you submit the form with invalid information (e.g. blank email address), you’ll notice that the cart URL changes. The URL now ends in “create_order”:

Shopify checkout URL for populating form fields

 Now you can see that my email address is pre-populated in the form. You can pre-populate all form fields if you wish. In our case we are only pre-populating the email field and using CSS to hide the table row. This effectively lets our users checkout without re-entering the email address we already know.

Alternatively, use Mechanize to automate creating Shopify customer accounts with passwords

Goal: Create Shopify customer logins linked to third-party user database

Shopify limitation: Shopify customer logins can only be created by inviting a customer (manually) via email, at which point customers can create a password.

Originally wanted to create a Shopify customer account for each user in our third-party database, but Shopify doesn’t let you create customer accounts through the API. For some reason the only way to create a Shopify customer account is to manually “invite” the user via email, which sends them a link to create a password. We ended up automating this process using Mechanize:

  1. Create a Shopify customer using the API
  2. Create a Shopify administrator account that only has access to Customers
  3. Mechanize logs in to shop admin with administrator account
  4. Mechanize retrieves the invite link for a customer
  5. Mechanize visits the invite link and fills out the form to create a password

Pretty simple. However, we felt that this was less reliable than the guest checkout method. This process would also sometimes take up to 7 seconds to complete, and we decided it would add more complexity to our application to deal with this latency, as well as making sure that our users would be logged in to their Shopify accounts behind the scenes reliably (read: posting their Shopify username and password to the login form on Shopify in a hidden iframe).

Download the mechanize script (mechanize_activate_customer_login.rb) from Github. It’s a Ruby script that you can run from the command line. It accepts several arguments and outputs how long each step takes. Make sure to install the Mechanize gem before running the script.

Some other convenient undocumented Shopify functionality discovered

We actually have our own custom cart on a website separate from Shopify. So, when users click the “checkout” button, we need to “copy” the items we have in our cart to the Shopify cart. To do this, we first need to clear their existing Shopify cart and then post all items to Shopify.

Redirect after clearing Shopify cart

To clear a Shopify cart, you visit the “/cart/clear” page. After the cart is cleared, it automatically redirects you back to the empty cart page:

However, if you want to send your user to a page of your choosing, all you have to do is add that page to the URL as a “return_to” parameter:

This parameter is documented for adding products to the cart, but not any other pages. Presumably, you can add the “return_to” parameter to more pages that perform actions within Shopify.

Projects Technology Web

Easily Transform JSON with json2json

I have been working on an auto-complete web service that searches Amazon’s Product Advertising API.  I built it in Node.js and using the APAC package made it really easy to query the API. The only thing that was extremely impractical was the JSON data returned by APAC.

Since Amazon’s API only returns data as XML, APAC uses xml2json to convert the XML to JSON. Unfortunately the resulting JSON is quite ugly. I wanted to be able to choose the data I needed and copy it to a new, clean JSON format. My solution was to create json2json.

json2json lets you create a template that describes how to transform the original JSON to a new structure. I wrote the Node.js package and example template in CoffeeScript because it has a much cleaner and simpler syntax than JavaScript. However, it is extremely simple to convert to JavaScript (click on “Try CoffeeScript”) and can easily be modified for use in a browser. Check out the (crude) documentation and example files and let me know what you think.

Projects Technology Web

FBAPI.js: Facebook JavaScript SDK Simplified

I’ve been working on Rembly using several javascript libraries: Spine.js, Mustache.js, ICanHaz.js, and Facebook’s JavaScript SDK. These have made application development easier, but not always easy enough! I wrote previously about my enhancements to ICanHaz.js for loading Mustache templates. This time around, I wanted to use Facebook’s JavaScript SDK with less “overhead” and a simplified API.

I created FBAPI.js to handle the setup requirements that Facebook’s SDK requires, such as adding a “root” tag to the page before loading the SDK. Now FBAPI.js takes care of all the SDK requirements and lets you use the Graph API without worrying about the overhead. FBAPI.js adds helper methods for event binding and retrieving user data.  However, the best part of FBAPI.js is that you don’t have to wait for the page or javascript dependencies to be loaded before you can start using it! All methods use promises and callbacks. This lets you run your scripts in any order you want!

Check out the Github repository and let me know what you think!

Technology Web

Easily Overload JavaScript Functions with Optional Parameters

I just answered a Stack Overflow question from a couple years ago titled “Handling optional parameters in javascript” and figured I’d write about my solution here.

I’ll start by saying that the easiest way to handle optional parameters in javascript is to use an “options” object that allows a function to be called with as many or as few parameters (arguments) as you wish.

function displayOverlay(options) {
  if (options.alert) { 

However, if you need to use individual parameters, i’ve created a utility that acts as a proxy and lets you strongly type values.  It looks like this:

function displayOverlay(/*message, timeout, callback*/) {
  return proxy(arguments, String, Number, Function,
    function(message, timeout, callback) {
      /* ... your code ... */

I call my proxy arrangeArgs(). Here’s a clearer explanation of what’s going on:

function displayOverlay(/*message, timeout, callback*/) {
  //arrangeArgs is the proxy
  return arrangeArgs(
           //first pass in the original arguments
           //then pass in the type for each argument
           String,  Number,  Function,
           //lastly, pass in your function and the proxy will do the rest!
           function(message, timeout, callback) {

             //debug output of each argument to verify it's working
             console.log("message", message, "timeout", timeout, "callback", callback);

             /* ... your code ... */


I created the arrangeArgs() proxy to handle optional parameters for you.  It works nicely.  The code is in my Sysmo.js utility library on GitHub. Let me know what you think!

Projects Technology Web

A Super-Charged Version of ICanHaz.js

I’ve been working on Rembly, which uses Spine.js as the core piece that ties all the functionality together.  I decided to use Mustache.js for my HTML templates.  And finally, I chose ICanHaz.js as a simple and lightweight way of managing my HTML templates.

Although ICanHaz.js is a great start, managing my HTML templates became unwieldy because I started having little templates everywhere.  Each part of a page that is dynamically updated needs to be broken out into its own template.  When you’ve broken a web page into small parts, it’s hard to keep track of what it looks like when put back together.  It also becomes hard to create the correct CSS styles when you lose track of the HTML hierarchy.

This lead me to enhance ICanHaz.js with a ton of new features.  The primary one being nested templates, which allowed me to keep my full HTML page template in tact, while designating specific HTML tags as “sub templates” or partials. You can also specify additional templates to load and replace script “include” tags with the loaded HTML.

Check out my fork on Github for more information about how to use my enhanced version if ICanHaz.js. Make sure to look at the javascript comments for details. And let me know what you think.

Projects Security Technology Web

Announcing the ThreatSim Beta

I’ve been working with Stratum Security for the past couple of months on ThreatSim (@ThreatSim), which we are happy to announce to the world today!  ThreatSim is a web-based phishing attack simulator to help companies assess how vulnerable their network and internal assets may be to phishing attacks.  Not only does ThreatSim track who is clicking on phishing emails, but we’re also making an exfiltration agent available, which simulates transmitting sensitive data from the local network out to the internet.

Check out the website at, follow us at @ThreatSim, and check out the conversation on Hacker News.

Creativity Mood

Draw Happy

Draw what makes you happy?  What a great idea.  Happiness is redefined by everyone who experiences it.  To define happiness can be as simple as expressing what gives you joy.  Think about what makes you happy and draw it!

Draw “happy.”  Join the global art project at Draw Happy and share your happiness with the world.  Don’t know what makes you happy?  I guarantee the creative expressions at Draw Happy will remind you and hopefully bring a smile to your face.



So many innovative ideas come about by accident.  They come about by experimenting and being creative in your own unique ways.  Just playing around with ideas can lead to exciting new discoveries.  Schools don’t teach us to play.  They don’t encourage us to explore.  They don’t even really prepare us for tomorrow, but rather prepare us for test taking.

Robert Scoble presented at TEDxBloomington about “play” and what wonderful discoveries and inventions it can lead to. Coincidence plays a big part, but it’s primarily curiosity.

Check out the great examples of “play” and what comes of it!


Your Own Scale for Measuring Success

I’ve read Seth Godin’s books over the past couple of years, and he consistently has great insights into marketing and human behavior.  His post about “Originality” was very timely, coinciding with Austin Kleon’s post on the same subject.  His way of thinking is a parallel to what I aim to apply to my own life, and the perspective I want to share with others.

People have different motivations to achieve success and how they measure their achievements.  Our society puts a lot of value on becoming famous and achieving recognition by the main stream media.  It’s sometimes dumbfounding to observers when someone with exceptional talent and influence does not follow the “trendy” routes their peers may follow to become recognized.  Seth Godin talks about chef Charlie Trotter, whom he calls “a pioneer in modern cooking”.  Someone who is achieving greatness and measuring success in his own way.

Fame and fortune are not what motivate all of us.  Seeing how you affect the people around you can be gratifying enough.  “Dancing faster than ever, but why?” highlights Charlie Trotter as one of these people, and it’s refreshing to have an influencer such as Seth Godin reminding us that we don’t have to measure success the way others may want us to.