About Arun Sivashankaran

I'm a tech entrepreneur who has been building, measuring and selling consumer and enterprise websites for years. Over the course of my career I've helped companies large and small increase revenue and engage customers as a manager, advisor and consultant.

Optimizely-CLI: Faster test iteration for developers

Today I’m going to take you through a really brief walk through and introduction of Optimizely-CLI.

A little bit of background: we at FunnelEnvy write a lot of tests in Optimizely. We love it as a platform. But we’re also software developers and engineers, and so we like to write our tests directly in JavaScript code. It’s much faster for us, and we’re really looking for ways to be able to use a lot of the tools that we use for software development in order to write tests.

We found as we built out the CLI, which stands for Command Line Interface, that being able to do a lot of the test development and debugging locally on our own machines was actually much faster. Optimizely-CLI allows you to do that, and then when you’re ready to push it up to Optimizely, you use the Optimizely API to push your test up.

You certainly do have to do a lot of the final tweaking and edits and QA directly in Optimizely, but the largest component of tests, especially more complex tests, not just changing around a word or two or a color but tests that are significantly more involved, is writing the code and is developing it. So, hopefully, if you’re a developer writing Optimizely tests, this can have value for you.

So, let’s get into it. I’m going to skip past the installation of the optcli itself. It’s pretty straightforward. It is built on Node, so you’re going to have to have Node installed locally. It’s delivered as an NPM package, so you’re going to have do an NPM install of Optimizely-CLI. Documentation is on our page, so feel free to take a look.

Once you have installed Optimizely-CLI, what you’ll want to do is create a new directory for your project. You’ll see that this one here is empty. The first thing you’ll want to do is initialize your project. The optcli init command is the one you’ll want to use for that, and you need to specify your project ID. The easiest way to find that is to head over to Optimizely and your project and go to the project settings tab, where under snippet implementation you will see your project ID.

Now, if I look at it, optcli has created a project.json file. An empty project is, obviously, not very useful, so we want to create our first experiment. Let’s say we wanted to do a headline test on the FunnelEnvy home page. To do that, you’ll want to use the optcli experiment command which takes several arguments including the directory that you want to create the experiment in, the description of the experiment, that’s what’s going to show up in Optimizely in the dashboard and in several other places, as well as the default editor URL.

When you open it up in Optimizely, this is the page that will open. You’ll see that it has created the folder and in there is the experiment.json file that describes the experiment.

Next up, we’ll want to create a variation within that experiment. To do that, we’ll use the optcli variation command, which in a similar way takes the folder name of the experiment, the folder name of the variation, as well as the variation description.

Let’s take a closer look at the files that optcli created. As you’ll see, here’s the project. Underneath there, we’ve got the headline test experiment, and underneath there we’ve got a variation called headline test. By default, optcli will include a variation.js as part of the variation, a global.css, as well as a global.js. This corresponds to the objects within Optimizely, the variation JavaScript as well as the global CSS and the global JavaScript that take effect for all variations within the project. The JSON files, as I mentioned, in here contain metadata about the variation project and experiment.

So, let’s add a little bit of code to our variation. As I said, we’re doing some headline testing. We’re going to make some changes to this headline at the top of the FunnelEnvy home page. So, we can do that right in the variation.js on our local directory, much as you would in the Optimizely edit code window, and we can add in a bit of JavaScript right there, changing the H1 tag to “Welcome to FunnelEnvy.”

Next, we’re going to want to style that, and so we can add some CSS right in the global CSS. Keep in mind that, just as with Optimizely, the global CSS will take effect for every variation. So, let’s make the font bold. Save that.

Then, just say for illustrative purposes, put a function in the global JavaScript file. And in order to call this function, I’m actually going to go back to the variation.js and call it right from there.

So, now my initial code for the variation is set. I’ve got code in the variation JavaScript, styles in the global CSS, and a function in the global JavaScript. Once you’ve coded your variation, optcli will let you host that variation locally and preview, debug, edit, and finalize the code for the variation before you push it up to Optimizely. That lets you make sure everything is right, iterate quickly on the code, and then make the final adjustments once you’re in the Optimizely environment.

Let’s take a look at what that looks like. The way to start the local server is to use the optcli host command and specify the path to the variation. By default, optcli uses port 8080. You can change that using the command line argument. And, by default, it serves over HTTP. You can also specify minus S for SSL if you are testing on HTTPS pages. If you point your browser to the URL from the console output, you’ll see that Optimizely-CLI has some instructions as well as some status and URLs for you.

On the left hand side, there are some one time setup steps. You only need to do this once, the first time you do the local hosting and script injection with Optimizely-CLI. And on the right, you’ll see the currently served experiment and variation as well as URLs to be able to view the variation on the editor URL that you specified in the experiment.

The script injection does require either the Tampermonkey Chrome extension, if you’re on a Chrome browser, or the Greasemonkey Firefox add on. In this case, I’m on Chrome, and I have installed the Chrome extension already. So, I’ll skip ahead to step two.

Step two is installing the Optimizely-CLI user script. This is generated by optcli. It’s a relatively simple script that will inject the locally served variation and CSS, locally served JavaScript, into the page by dynamically creating elements. In this case, I’ve already installed it. I’m going to hit reinstall anyway and then come back to the tab that I was on.

Now, Tampermonkey is installed on my browser. The injection script is running. I can now actually view the variation that I created. If you take a closer look at that script, optcli uses the optcli activate parameter to determine whether or not to inject the script. So, you can actually run your variation script on any URL just by appending this parameter.

Let’s go ahead and click on the link and see the variation. You can see here that this was the function that was executed in the global JavaScript, and per the variation code, the headline has been changed and the font style has been bolded. Now if I want to go back and make a change, it’s very simple to do. If I wanted to change the text in my variation to “Welcome Fellow Optimizer”, I change that locally, save the file, go back to the browser, hit refresh, and the change is done. And let’s say I wanted to get rid of the alert. It never made a lot of sense anyway. I take that out, refresh the page again, and it’s gone.

This lets you iterate really rapidly because you are doing it locally and then can therefore move a lot faster and get your code right. Let’s say I am happy with my code for the variation now and I want to send it up to Optimizely. Go back to the command line. We can stop the local server and run the optcli push experiment. It’s important to remember that you need to have pushed the experiment before you can push any of the variations underneath it. To push a variation, you just push experiment and specify the directory of the experiment.

If you haven’t entered your Optimizely API token, you’ll need to do so now. You only need to do this once on a project. It will store this locally. I’m going to enter my token now.

So, the token’s been set, and it has created a remote experiment in Optimizely. So, let’s head back over to Optimizely. Looking at the dashboard, refreshing the page, and I can see the headline test has been created. Let me push up the variation now that I created. To do that, you use the optcli push variation command specifying the path to the variation which is obviously underneath the experiment locally. And I didn’t need to enter my token, because I’ve entered it once and you can see that the variation has been created.

One final note on both the push experiment and push variation command. You can run these multiple times. The first time for both it’ll create either an experiment or a variation. But, if you make some local changes and run it again, it will actually update the code for those variations.

That’s how you can keep your local code in sync with what’s in Optimizely. So, if I go take a look in Optimizely, you’ll see that the headline test one experiment has been created as well as the change headline variation. By default, Optimizely will create a variation number one. You can remove that if you choose.

I hope you enjoyed this brief introduction to Optimizely-CLI. There are several other features there that I haven’t mentioned, so certainly feel free to give it a spin. If you have any feedback, we’d love to hear it. Feel free to submit issues or, better yet, put requests on the GitHub page. Thank you.

By |2015-02-04T13:17:43-08:00February 12th, 2015|Testing|0 Comments

How One Email Marketing Technique Can Help Recover $2.5 Trillion Dollars for Retailers

There’s one trend in email marketing that every marketer and entrepreneur needs to pay attention to: cart abandonment emails or retargeting. While the topic is always relevant, it should be top of mind for retailers moving into holiday marketing mode. Retargeting is a follow-up marketing strategy that goes after the customers who loaded items into their shopping carts on eCommerce websites, and then left without completing the purchase. The reasons for abandonment vary, from frustration with the checkout process to real-life distractions that come up while the customer is trying to finish processing a sale.

On average, across the eCommerce industry, The Baymard Institute estimates that 67% of attempted purchases end in abandonment. Other sources suggest that numbers are rising, as high as 74%. The real costs of these abandonments to the economy’s bottom line are all over the place, but one recent Inc piece estimated it was in the region of $4 trillion dollars. But according to Forbes, a startling 80% of big retailers are doing nothing about it. Even though the numbers aren’t 100% in agreement across sources, one thing is clear: they’re high, and there is a tremendous opportunity for businesses to use retargeting technology.  (more…)

By |2014-11-05T23:25:10-08:00November 6th, 2014|The Funnel|2 Comments

The Unauthorized Guide to Using Optimizely with Marketo

At FunnelEnvy several of our clients use Marketo. It’s a great solution for marketers to consolidate lead data and automate many of the aspects of marketing campaigns, including nurture campaigns and landing pages. Marketo offers a lot of capability for non-developers, which is probably why it’s seen such rapid adoption especially for lead generation use cases. Marketo offers A/B testing capability for landing pages, but we generally prefer Optimizely because it offers more flexibility and power when it comes to page testing, and we can also use it on the non-Marketo web properties. Both Marketo and Optimizely offer limited documentation on the topic, but I thought I’d share some of the hard-learned lessons (mistakes) that we’ve made with the hope that you can avoid them in the future. Together they’re two of the top tools in the market to drive conversions and personalized experiences.

A word of warning – some sections of this post get technical. Feel free to pass along to your developers or technical optimization team.

(more…)

By |2014-12-01T09:56:12-08:00October 15th, 2014|Conversion Rate Optimization|0 Comments

Mobile Path to Purchase Study: Optimizing the Mobile Funnel

Intent to purchase: it is perhaps the least discussed and most important aspect of conversion optimization. Ultimately, you focus on improving your website conversions to attract leads, build better relationships with customers, and most of all make sales. The ability to identify when a customer has a firm intent to purchase gives you a clear advantage. But how can you identify the intent to purchase and take steps to foster it in today’s mobile environment? One new report is offering powerful insights for marketers, entrepreneurs, and data analysts thinking about this issue.

Telmetrics recently released their third annual Mobile Path to Purchase study, in partnership with XAd and the Nielsen Company. Each year, the study seeks to find data, trends, and patterns on what factors are influencing mobile device use in the United States. By extension, the analysts are able to identify consumer mobile usage behaviors and the implications for intent to purchase.

The results provide specific insights into a range of verticals from retail to travel. But there are major takeaways that can inform any mobile strategy.

(more…)

By |2014-09-04T12:42:49-07:00September 5th, 2014|The Funnel|0 Comments

How KPIs Can Focus Your Conversion Testing

In a recent piece featured by MarketingProfs, Laura Patterson explored the notion of three categories of key performance indicators (KPIs) that determine success in online marketing endeavors: outcome metrics, performance metrics, and process metrics.

This is a useful framework for developing an approach to decoding and prioritizing the KPIs that you select for your conversion optimization efforts as well – whether you’re developing tests to help improve lead generation, ecommerce sales, or subscription performance. Here’s a closer look at why KPIs matter in your conversion efforts and how selected strategies can help you uncover the KPIs that have the most impact on how you do business.
(more…)

By |2014-08-17T20:45:48-07:00August 19th, 2014|Strategy|2 Comments

Facebook and OKCupid Experiments: Ethics, Conversion Testing, and UX Design

The headlines have recently been flooded with the news that Facebook and OKCupid have been conducting experiments on their users in an effort to collect data and improve the overall user experience.

Analysts and commentators from the public at large have raised the same concerns: is this ethical? Does running certain types of testing violate user trust, especially when the impact is carried off your website and into the “real world?” Where do companies draw the line?

While you shouldn’t let fears dominate your testing agenda, it’s helpful to have a clear idea of where your company stands on these issues and how that impacts what you disclose to customers and website visitors. Here’s a closer look at some of the ethics of testing and what these recent case studies can teach us.
(more…)

By |2014-08-16T15:30:04-07:00August 12th, 2014|Testing|5 Comments

5 Tips to Increase Customer Loyalty in the Age of Social Media

Customer loyalty is an important metric of your business’ success. It gives you insight into issues ranging from whether you’re serving the right customers, to the quality of your products, to the level of your customer service. Loyal customers also have the potential to dramatically increase your profits over time. But loyal customers don’t just happen; they’re cultivated as part of a conscious process that’s built into your business’ DNA.

It’s up to you to use all the tools at your disposal to make this happen. Social media has opened new channels of communication, while also raising the bar on accountability for businesses. Here’s how to make the most of this opportunity, by applying good data analytics and testing models, and avoiding common social media pitfalls.

(more…)

By |2014-07-02T22:34:17-07:00July 3rd, 2014|Digital Marketing|4 Comments

Mapping Your Funnel: How to Find the Microconversions that Matter

If you ask the majority of firms doing online marketing what their most important conversion goals are, most will tell you that it’s to make a sale. A few firms that are tuned into the broader marketing discussion might tell you that they’re focused on building their email lists or customer contact database. But far fewer will tell you that they are focused on reducing friction along the buying funnel.

Yet for most businesses, it’s understanding the universe of small conversions that lead up to the sale – and eliminating obstacles along the way – that ultimately yields the best results. Here’s a closer look at microconversions and how they affect your firm’s performance.

(more…)

By |2014-06-27T09:34:55-07:00June 30th, 2014|The Funnel|2 Comments

How to Collect The Data You Need to Develop a Winning Conversion Optimization Strategy

We all want better results.

Who couldn’t use more visitors turning into leads, and more leads turning into paying customers?

You have the power to create those results for your business. But in the words of Peter Drucker, “you can’t manage what you can’t measure.”

That’s why conversion-rate monitoring and optimization are vital. Turning to data to understand what’s working well and what could be improved puts you in the best position for sustained success.

Conversion Rate Optimization is an investment. Getting the best possible ROI on that investment requires finding and gathering the right kind of data. The higher the quality of the data you collect, the better insights you’ll gain from testing and evaluating the results.

Collecting high-quality data isn’t as difficult as you might think. Keep reading to see how to do it.

(more…)

By |2014-06-08T23:44:27-07:00June 9th, 2014|Strategy|2 Comments

5 Tactics for Increasing Your Business’ Word-of-Mouth Marketing Traffic

One recent Neilsen study revealed that 92% of consumers trust referrals from family or friends more than any other type of advertising or referral. Word-of-mouth (WOM) marketing is often considered the marketing gold standard in lead generation and online marketing. For every lead or prospect you can work to attract through a savvy website or clever content, a happy customer can effortlessly send you numerous referrals that become profitable customers for life. Yet many entrepreneurs and small business owners believe that word-of-mouth marketing is something that occurs by accident or at least at the will of customers.

(more…)

By |2014-05-18T22:47:34-07:00May 19th, 2014|Digital Marketing|3 Comments
Go to Top