Tagged: plan Toggle Comment Threads | Keyboard Shortcuts

  • mitcho (Michael 芳貴 Erlewine) 11:33 pm on August 18, 2010 Permalink | Reply
    Tags: meeting, plan,   

    Had a great conversation with Joen, wher… 

    Had a great conversation with Joen, where walked through the process of experiment creation together. Here’s my notes from the meeting:

    • switch to single self-closing [ab /] shortcode
    • A/B shortcode feedback in visual editor
    • add A/B button

    – look at ratings plugin
    – test with TinyMCE advanced, etc.

    • variant viewer: close button
    • help / what’s this links, hover links
    • think about first run experience
    • complement “A/B testing”, or alternative wording for marketing
     
  • mitcho (Michael 芳貴 Erlewine) 3:21 pm on July 26, 2010 Permalink | Reply
    Tags: , plan,   

    This week: I’ll be continuing to document the development API, and will create a few other custom metric/variant types to further dogfood the API.

    The first example I hope to build is a plugin which encapsulates the workflow of A/B testing different titles on posts, and then automatically selecting the winner, à la Huffington Post.

    Over the weekend I’ve thought more about the metric and variant type API’s. Right now my approach has been to require that these “plugins” be object-oriented: defining a class which extends ShrimpTest_Metric and ShrimpTest_Variant, respectively. But the actual mechanism by which these hook into different parts of ShrimpTest are using the WordPress-standard filter/action model. I’m wondering whether, perhaps, instead, I should define some particular method names which, if defined by the class, are called at particular points in the life-cycle of the experiment.

    For example, when a new experiment is called, the extra data for its metric from the form is passed through a “shrimptest_save_metric_<metric name>” filter. Instead, perhaps I should check whether the metric has a “save_metric” method, for example, and if there is, calling that method.

    If anyone has any thoughts on whether one pattern would be better than the other, I’d love to get such feedback. 🙂

     
    • mitcho/芳貴 4:35 pm on July 26, 2010 Permalink | Reply

      Just had a good conversation with Joseph Scott on the power of the filter/action model, and I’m now thinking I may actually dissolve this requirement that custom metrics + variant types be written in an object-oriented fashion.

      Instead, I could expose functions shrimptest_register_metric_type and shrimptest_register_variant_type with argument structure similar to register_post_type.

    • Nabha 3:01 pm on July 30, 2010 Permalink | Reply

      Looking forward to the API documentation. I was thinking again this week about when to start using ShrimpTest on our own site.

      • mitcho/芳貴 12:01 am on July 31, 2010 Permalink | Reply

        When I say “API docs”, I’m talking about custom metric types and custom variant types. Using ShrimpTest and doing selection and metric-recording via PHP is already fully supported… just try downloading a current version, go the ShrimpTest experiment listing to “add new”, and use the “manual” variant and metric types, and you’ll see some sample PHP on the right. There’s a file with example PHP code in the examples directory as well.

  • mitcho (Michael 芳貴 Erlewine) 12:36 am on July 19, 2010 Permalink | Reply
    Tags: plan, ,   

    Last week: successfully implemented a normal distribution CDF (culmulative distribution function) in PHP so that we now have information on confidence intervals.

    I also took time to refactor the core code, moving the experiment/db-management code into a separate class (#1146). ShrimpTest’s core singleton objects are now in an MVC configuration.

    I also spoke to Alex briefly on IRC regarding unit testing, as I’d like to incorporate some tests into my development. I hope to try out the WordPress-Tests framework and write some tests soon #1161.

    This week: This week my focus will be slightly different, as I have three WordPress-related talks in the next week:

    1. ShrimpTest: A/B Testing for WordPress at the Bocoup Loft in Boston, Wednesday July 21st, 6:30pm
    2. Set your sites on WordPress 3.0! at the MIT WebPub interest group, Thursday July 22nd, at Noon
    3. A shorter, (more) casual version of ShrimpTest: A/B Testing for WordPress at Crowd Favorite in Denver, Monday July 26th. This event isn’t public (I think) so let me know if you’re in Denver and would like attend.

    In the next week I’ll be spending a decent amount of time preparing for these talks and recovering learning from the feedback I receive on ShrimpTest. I’m looking forward to showing off this project and seeing what people think! 🙂

     
  • mitcho (Michael 芳貴 Erlewine) 9:43 am on July 12, 2010 Permalink | Reply
    Tags: , plan,   

    Had a great time in Whistler with Mozilla but now I’m back! Thanks to all who commented last week on my June progress report!

    This week: further refining the UI and making the statistics easier to understand.

    As part of this, I hope to take a little time playing with some other A/B testing frameworks as well. I’ll also add significance intervals to results, based on the z-scores.

    I’ve received a couple new tickets from those who have tested ShrimpTest in the past week, and I hope to respond to those quickly as well. I’m looking forward to more tickets and feedback! 😀

     
  • mitcho (Michael 芳貴 Erlewine) 11:24 pm on July 5, 2010 Permalink | Reply
    Tags: , plan,   

    A quick reminder that I will most likely not be working on ShrimpTest for the rest of the week, as I’ll be at the Mozilla Summit!

     
  • mitcho (Michael 芳貴 Erlewine) 11:32 am on June 28, 2010 Permalink | Reply
    Tags: plan,   

    This week:

    • WP Super Cache integration (rolled over from next week)
    • Experiments screen UI refinements
    • Put together a screencast: “ShrimpTest: a month of work” or some such

    Note: next week: I’ll be at the Mozilla Summit for most of the week, so will probably only be putting a day’s worth or so of work in next week. FYI.

     
  • mitcho (Michael 芳貴 Erlewine) 1:27 pm on June 21, 2010 Permalink | Reply
    Tags: plan,   

    This week:

    • More work on the Add New Experiment screen and API for custom metrics and variants.
    • Wrapping up the first custom metric type: conversion.
    • Getting things to work with WP-SuperCache (spoke to Donncha this morning and got some tips)
    • (Hopefully this week:) starting on the first custom variant type: simple string substitutions/variants via [ab] shortcode.
     
  • mitcho (Michael 芳貴 Erlewine) 11:13 pm on June 7, 2010 Permalink | Reply
    Tags: plan,   

    This week: I’m reading up and reviewing my stats, then starting to create some UI for creating new experiments and viewing the aggregate results. This will still be low level this week… you’ll still need to add some PHP to your theme or elsewhere to get the variants and metric recording to work.

     
    • mitcho/芳貴 11:14 pm on June 7, 2010 Permalink | Reply

      Also, FYI world: traveling to Chicago Friday for my girlfriend’s graduation. 🙂

c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel