Tagged: status Toggle Comment Threads | Keyboard Shortcuts

  • mitcho (Michael 芳貴 Erlewine) 11:52 pm on September 4, 2010 Permalink | Reply
    Tags: , status   

    Inline documentation: complete! 

    Today, after a brief hiatus preparing for going back to school, I’ve completed and committed all inline documentation in the ShrimpTest source. This is all being done using PHPDoc format, and I also committed the PHPDoc documentation to svn as well. Feel free to check it out!


  • mitcho (Michael 芳貴 Erlewine) 11:27 pm on August 29, 2010 Permalink | Reply
    Tags: , status   

    Now that the experiment duration (sample size) and notification features have been implemented, ShrimpTest is feature-complete! Download it now!


  • mitcho (Michael 芳貴 Erlewine) 11:03 pm on August 29, 2010 Permalink | Reply
    Tags: status   

    Added some additional UI polish: start and finish times are now consistently recorded and displayed, and messages like “Experiment activated.” are displayed:

  • mitcho (Michael 芳貴 Erlewine) 7:59 pm on August 29, 2010 Permalink | Reply
    Tags: notification, status   

    Just committed the notification plugin that I’ve been working on for a while. This functionality was blocking on the experiment duration system. When an experiment stat is computed and trips the experiment duration line for the first time, the shrimptest_experiment_duration_reached action is triggered, which is used by this notification.

    Along the way, I had to modify a variety of things in the Model so that arbitrary data can be added to experiments and variants (such as, in this case, the notification emails) and can be updated correctly.

  • mitcho (Michael 芳貴 Erlewine) 12:55 am on August 28, 2010 Permalink | Reply
    Tags: status   

    Just committed UI for specifying/calculating experiment duration:

    The idea is that, as I mentioned last week, there are statistical problems with not specifying a sample size (i.e. an experiment duration) ahead of time. This UI simply asks the user what the “detection level” for this metric is (here, 0.01, in other words, 1% differences in conversion rate) and computes an experiment duration based on that, automatically.

    Doing this in the general case is hard, as we need to know the variance of the metric we are using. However, ShrimpTest can take advantage of the fact that our goal metrics are organized around “metric types”, “conversion” being one of them. Because conversion events are Bernoulli trials, their maximum variance is known, enabling this calculation. For custom (manual) metrics, an input box is shown to ask the user to specify a variance, if they would like to take advantage of the experiment duration calculation.

    This experiment duration value will be used in calculating when an experiment can be completed, enabling notifications when experiments are over without just waiting on a significant result.

  • mitcho (Michael 芳貴 Erlewine) 6:34 pm on August 24, 2010 Permalink | Reply
    Tags: , status   

    I added an A/B shortcode generating button to the plain text editor:

    The code for the TinyMCE version is also mostly there, though it’s not working and will need some debugging.

    Also, as seen in the video, the format of the A/B Shortcode itself has changed, following my discussion last week with Joen. Now instead of the control value being the inline content, it is also a shortcode attribute like the other variants.

    All these changes have been committed. Please try it out!

  • mitcho (Michael 芳貴 Erlewine) 11:33 pm on August 18, 2010 Permalink | Reply
    Tags: meeting, , status   

    Had a great conversation with Joen, wher… 

    Had a great conversation with Joen, where walked through the process of experiment creation together. Here’s my notes from the meeting:

    • switch to single self-closing [ab /] shortcode
    • A/B shortcode feedback in visual editor
    • add A/B button

    – look at ratings plugin
    – test with TinyMCE advanced, etc.

    • variant viewer: close button
    • help / what’s this links, hover links
    • think about first run experience
    • complement “A/B testing”, or alternative wording for marketing
  • mitcho (Michael 芳貴 Erlewine) 8:14 pm on August 13, 2010 Permalink | Reply
    Tags: , status, W3 Total Cache   

    Note: The latest trunk version of ShrimpTest now has support for W3 Total Cache (in “disk (basic)” mode) as well as WP Super Cache (in “half-on” mode), using the same single file.

  • mitcho (Michael 芳貴 Erlewine) 8:57 pm on August 9, 2010 Permalink | Reply
    Tags: , , status   

    Today: Worked on getting ShrimpTest to work correctly with the W3 Total Cache plugin, with limited success. Then started laying out ShrimpTest documentation on this site, and starting to fill in various sections. The first section, an introduction to A/B testing, is nearly complete.

    Also, reading about the computation of experiment sample size in advance.

  • mitcho (Michael 芳貴 Erlewine) 12:34 am on August 7, 2010 Permalink | Reply
    Tags: , issues, status   

    If any of you tried to use ShrimpTest in the past but discouragingly were given a “the ShrimpTest model has not yet loaded” error when activating the plugin, fear not! Thanks to the help of a few kind folks on twitter, I’ve been able to resolve this issue. Try downloading the latest development version:


    Thanks to Eric and Erik who helped report this issue! 🙂

Compose new post
Next post/Next comment
Previous post/Previous comment
Show/Hide comments
Go to top
Go to login
Show/Hide help
shift + esc