For growth engineers that code Optimizely experiments, we’ve released the tool that helped us increase test develoment velocity by 300%.

We’re huge fans of Optimizely at FunnelEnvy, and deliver a lot of tests with it for our customers. As CRO specialists we know that iterating faster is really important for delivering results. As engineers we’ve found that the best way to do that is to write our tests directly in Javascript.

Of course Optimizely lets you code in Javascript through the UI but since we’re stubborn engineers we wanted to use the software development tools that we’re used to. We also wanted faster debug cycles and to be able to publish direcly to Optimizely with a simple command. That’s why we built Optimizely-CLI (optcli) that our team uses (dogfoods) every day. It’s had a huge impact on our test development velocity so we’re releasing it for other developers who are building tests.

• This is primarily for test development (code) – You’ll obviously still have to come up with your own great hypotheses and prioritize them. Settings like audience / URL targeting, traffic allocation and activation mode will still have to be managed through the Optimizely interface.

• Optimizely-CLI is under active development – As an internal tool we could live with some workarounds. You may see behavior, messages and code that are suboptimal. Please report them so we can continue to improve this. Although we use it all the time there are certainly no guarantees made or implied.

• It’s made for developers – You’ll need to be comfortable with the command line and able to write code to get value out of this. Also we don’t use Windows so we’re really not sure how well it works in that world – issues and pull requests along those lines are welcome.

• Code evaluation that similar to (but not exactly) Optimizely – Optimizely does some performance optimization on the variation code that we don’t exactly mimic. Although we’re pretty close when you’re running locally all of your code will be executed at DOM ready. In many cases this doesn’t really matter for the purposes of development but it’s something you should be aware of.

Optimizely-CLI uses local files for the experiment and variation code (CSS & javascript). These include the variation javascript (variation.js) and experiment global.css and global.js. Variations, experiments and projects are in a nested directory structure which mimics Optimizely’s structure. Metadata for these elements are stored in json files – specifically project.json, experiment.json and variation.json files. Since everything is on the local filesystem you can obviously your favorite editors and source control tools to write, manage and share everything. A typical project file structure will look something like this:

Optimizely-CLI includes a command line executable, optcli. Optcli will create the local project experiment and variation structure for you, host the local variation for debugging, and publish to Optimizely.

We’ve found that injecting the experiment and variation code directly into the local browser makes for faster development & debug cycles. optcli includes an express.js based server that will compile and host a variation locally, and will generate a GreaseMonkey or TamperMonkey compliant user script to inject the CSS and JS into any page. Make a code change and refresh the browser – your variation changes will be immediately applied without restarting the server.

Targeting a page served over https? Not a problem! Optcli includes a self-signed certificate and can host over ssl with the -s option.

Script injection works by writing CSS and JS elements into the page, and targets any page which has an optcli=activate parameter. You can therefore run the variation code against any page by appending this parameter.

The variation.js and global.js files can contain embedded javascript (ejs), and are compiled before local hosting or being sent to Optimizely. That means you can use iterators or other logic expressions directly in your template.

As part of the of EJS templates, you can create an assets.json file in your experiment directory with key / value pairs. These will be available for interpolation when the javascript files are compiled. We find this handy when we write a test where we have to replace lots of images.

With the push-experiment and push-variation commands optcli lets you push your code directly to Optimizely via the REST API. You’ll have to have an API token with the appropriate permmissions issued for the project of course. Once the code is in Optimizely you’ll want to go in and adjust the other test parameters (targeting, traffic allocation, activation conditions, etc) per your test requirements.

Optcli is a node.js project and command-line executable. As such you’ll need to have node.js installed. To install optcli via npm:

npm install -g optimizely-cli

This will install the optcli executable onto your system.

optcli -h

View a list of optcli commands and options

Before you create an experiment or variation you’ll want to initialize a project. If have an existing project and have been issued an API key you can use the remote (-r) option, and if not just create it locally. You’ll need to specify the project id (required):

optcli init [options] [project_id]

Where options are:

• -r --remote – initialize from remote project

This will create a project.json in the current directory.

Create a local experiment under a project with the command:

optcli experiment <folder> <description> <edit_url>

Arguments:

• <folder> – The folder (directory) for the new experiment. Required.
• <description> – The experiment description that will show up in Optimizely. Required
• <edit_url> – The default editor url for the experiment. Required.

Create a local variation and scaffold the source files

optcli variation <experiment> <folder> <description>

Arguments:

• <experiment> – The directory or id of the experiment. Required.
• <folder> – The new folder (directory) that will be created for the variation. Required.
• <description> – The variation description that will show up in Optimizely. Required

Compiles the experiment and variation CSS / JS, creates a user script for injection and starts a local web server to host the files. Pointing your browser to the root URL (default http://localhost:8080) will show a page with installation steps and variation URLs.

optcli host [options] <path> [port]

Arguments:

• path – Path to the variation directory. Required.
• [port] – Port to host the server. Default to 8080.

Options:

• -s --ssl – host over https

Push (create or update) an experiment to Optimizely through the API. This will create a new experiment if it hasn’t been pushed before, or update the experiment global CSS & JS if it has.

optcli push-experiment <path>

Arguments:

• <path> – The path to the experiment directory. Required.

Options:

• -i --iterate – Push experiment and all variatons

Push (create or update) an experiment to Optimizely through the API. This will create a new variation if it hasn’t been pushed before, or update the variation JS if it has. Note that the experiment must be pushed via push-experiment before the variation.

optcli push-variation <path>

Arguments:

• <path> – The path to the variation directory. Required.

We would love to hear any ideas for new features and improvements for Optimizely-CLI. Here are some things we have on the short term roadmap.

• Pulling experiments / variations from Optimizely.
• Templating mechanism for complex layout tests

Bug reports / fixes, documentation updates, tests are all welcome and very much appreciated. Head over to the Optimizely-CLI Github page.