Taming Cybersource: the Cybersourcery Testing gem for Rails

Cybersource is a subsidiary of Visa, and is one of the largest providers of online credit card payment services. As any developer who has worked with Cybersource’s Silent Order POST service can tell you, it’s not the easiest service to work with. It provides a great deal of flexibility, but that comes at the cost of you having to write a good deal of your own code in order to use it. Setting up automated testing is also extremely difficult.

Last year I completed a Cybersource project for the University of Pennsylvania, and that project provided the inspiration for 2 Ruby gems, to simplify working with Cybersource: Cybersourcery, and Cybersourcery Testing. There’s also a demo project, so you can see an example of how to use them in a Rails project.

The readme files provide detailed documentation of their features and how to use them. So rather than repeat that information here, let’s take a look at why these gems are necessary in the first place. There’s a lot to cover, so I’ll discuss the testing gem in this post, and Cybersourcery in the next one.

Writing tests that can be repeated and automated provides benefits such as improving the design of your code (if you’re doing test-driven development) and catching regressions early (when changes to your code inadvertently introduce bugs). This can be challenging with 3rd party services, as we don’t want to call those services every time we run our test suite. VCR is a gem that helps with this problem: by recording requests and responses, it allows you to play back those responses in your tests, rather than making real-life calls in every test run.

Unfortunately, Cybersource makes this kind of testing especially difficult. There are 3 different servers involved in processing a transaction through Cybersource, and the key difficulty is that one of them is at a fixed URL that is not easy to swap out in the test environment. Cybersource calls this URL the “Customer Response Page.” It is saved on the Cybersource server as part of the merchant profile, so it cannot be updated dynamically. If you are a developer attempting to test Cybersource transactions, this diagram illustrates the scenario:

                     +                    +                   +
                     |     Developer's    |    Cybersource    |  "Customer Response"
     User's browser  |     test server    |    test server    |         server

    Request credit
      card form
          +----------->   Respond with
                        credit card form
     Submit form <-------------+
          +------------------------------>   Process and log
                                            generate "Customer
                                              Response" form
    Hidden "Customer <------------------------------+
    Response" form is
     submitted by JS
          +--------------------------------------------------->   Process submission;
                                                                 generate response page
        Display  <-------------------------------------------------------+
     response page

So, what the heck is going on here? The first few steps makes sense, but then when you submit the credit card payment form to Cybersource, things start to seem strange. What happens is that Cybersource sends a seemingly blank page to your browser. But it only appears for a second, as it contains a hidden form, which is immediately and automatically submitted to the “Customer Response Page.” This is the page where users are sent when transactions are complete. You provide the URL for this page when setting up your merchant profile in the Cybersource Business Center. This is a page you create and host yourself – you can use it to show users a “thank you” message, log information about the transaction, etc.

So why doesn’t Cybersource simply redirect to your response page after processing the transaction? Why this peculiar reliance on a hidden form? The reason is that conventional redirects use the GET method, which is meant for idempotent requests. An idempotent request is one that can be safely repeated, which definitely does not apply to a credit card transaction, or logging it. So Cybersource’s forms appropriately use the POST method, which is meant for non-idempotent requests. This is why, if you submit a POST form, and then click “back” in your browser, and try to submit the form again, your browser will warn you, and ask if you really want to submit the form again.

In the case of Cybersource, this is a thorny problem. Trying to do a POST redirect has issues, for these reasons. A redirect isn’t really appropriate anyway: the Cybersource server does some work when it receives the user’s credit card submission (charging the user’s card), and then your response page may also do some work when it receives the hidden form submission (such as logging details of the transaction). These are distinct activities, so – while having two forms may seem odd – it’s a viable solution. Cybersource came up with this before asynchronous requests were a common practice (which is a big part of the reason it’s harder to work with than newer services like Stripe).

The Cybersourcery Testing gem makes it possible to set up automated, repeatable tests in this complex environment. It provides a “translating proxy” server, running on Sinatra, which has middleware to manage the requests and responses between the 3 servers involved in Cybersource transactions. Middleware is especially useful in this situation, as it allows us to modify requests and responses before they reach the application’s code.

In order to explain how the gem works, let’s first take a look at its dependencies:

  • The Rack Proxy gem is a Rack app which allows you to rewrite requests and responses. It’s very open-ended and is designed to be subclassed, so you can implement rewrite rules appropriate to your project.
  • The Rack::Translating Proxy gem inherits from the Rack Proxy gem, and provides an implementation that’s suitable for the Cybersource use case. We need to provide it with a target_host URL, which indicates where the proxy server should redirect requests. We also need to provide it with a request_mapping, which indicates what strings to find in the requests and responses, and what to change them to. It uses a hash format, so that on requests, the keys are translated to the values, and on responses, the values are translated to the keys.

The Cybersourcery Testing gem inherits from the Rack::Translating Proxy gem, and implements the methods described above. Specifically:

For the target_host, we provide the URL of the Cybersource testing site. So if the proxy server is running at http://localhost:5556, and the target_host is https://testsecureacceptance.cybersource.com, requests to http://localhost:5556/some/path will be redirected to https://testsecureacceptance.cybersource.com/some/path. The gem also hooks into VCR, allowing us to record transactions that pass through the proxy server, for use in subsequent automated tests.

This is a simplified version of the request_mapping implementation, using hard-coded values for clarity:

def request_mapping
    # local test server                Cybersource's "Customer Response Page" URL
    'http://localhost:1234/confirm' => 'http://your-site.com/confirm'

A Cybersource transaction in this environment looks like this:

  1. The credit card form is submitted and the proxy server receives it.
  2. Based on the target_host, the proxy server passes the request through to the actual Cybersource test server. If the transaction was previously recorded with VCR, VCR will instead play back the recording of the transaction.
  3. Cybersource (or VCR) sends a hidden form back to the browser, which is automatically submitted via JavaScript to the “Customer Response Page” URL. The middleware’s request_mapping will rewrite the URL of the form’s action, causing the form to instead submit to the local test server.

The upshot is, the gem handles all this complexity so you don’t have to. By following the setup steps in the readme, you can get a robust test environment setup for Cybersource without breaking a sweat. The Cybersourcery Testing gem offers other features as well, such as reporting of undocumented Cybersource errors. Check out the README to learn more!

WordCamp Nashville, and 10 tips to keep your code clean

The Poka Yoke Design team came together at WordCamp Nashville yesterday, to officially launch the business. We’re very excited to get started, and WordCamp Nashville was the perfect place for it. Our co-founder Michael Gyura came from Memphis, along with Susan, our Director of Operations, and our Director of Support, Justin, lives in Nashville. I’ve flown from Philadelphia to Nashville every year since WordCamp Nashville started in 2012, to be a speaker. This year I gave a talk on Agile Contracting for WordPress consultants. I’ll write another post about my talk once the video is available.

Almost all WordCamp talks are recorded and posted to WordPress.tv, and the best ones are featured on the home page. My 2013 WordCamp Nashville talk on Clean Code was featured on the home page for over 5 months. In that talk, I pointed out that something most developers don’t realize in their work is that the ratio of time spent reading code vs. writing code is, on average, over 10:1. So when we rush to meet a milestone deadline of a project, rushing causes us to do sloppy work, which results in the project going more and more slowly over time. This is because the more we rush, the more tangled and harder to understand the code becomes. In the talk I outlined 10 straightforward practices that any developer can follow, to start developing the skills and self-discipline to keep code clean and easy to understand. The purpose isn’t to satisfy some abstract notion of perfection – it’s so you can work all the time at a reasonably fast, steady pace, and at the same time consistently deliver quality work to your clients.

Here are the video and slides from my Clean Code talk, as well as some tweets from people who were there.

Tweets about the 'Clean Code' talk at WordCamp Nashville 2013

We’re Baaaack!

After a brief (or not so brief, depending on you look at it) hiatus, Poka Yoke Design is back up and running!

Although we were gone for a while, we’ve been hard at work pulling together a team of crazy and devoted people to meet all your needs. You can see more about us in our bios, but here is a quick overview of the people you will be working with:

•    Michael Gyura is the president and co-founder of PYD. He’s a pro on WordPress and likes to maintain that he isn’t a hippie, despite how much he loves the outdoors.
•    Mike Toppa is CTO and co-founder of PYD. He’s a master coder, with over 20 years of experience, and is just as good at Nerf gun battles thanks to his sons.
•    Susan Gyura is director of operations. She is great at planning, implementing and managing programs for PYD and is glad that sandal-weather is here.
•    I, Rachel Shussett, am the social media and marketing director. Strategic planning and writing are my forte, and when that’s not happening I’m probably playing my guitar or running.
•    Kyra Dosch-Klemer is PYD’s lead designer. She is wonderful at crafting both traditional print and websites/apps that are unique and user-friendly. She loves to practice yoga and lives for media artwork.
•     Justin Near is director (or Queen, if you will) of support. She is available to PYD clients no matter the problem. Outside of the technology world, Justin also has a small business selling crafts that are handmade by homeschooled students and families in the Nashville area.

We are so excited to be back in the web designing game and can’t wait to show you some of our new and improved moves. Stay tuned for much more to come.