Tag Archives: programming

Tracking Email Open Time with PHP

Posted on by
3

Some time ago I had great aspirations of launching a web company that does email tracking and analytics. One of the things that I really wanted to figure out but wasn’t well documented on the web was how to track how long a user had a particular email open. When a company like MailChimp wants to track emails that they are sending out, they put a small image in the email called a “beacon”. When the user opens the email, the beacon image is requested from the server. The server sends the image, but not before it gathers information about the computer requesting it. The works great for checking if an email was opened, or what platform the person is on, but it doesn’t work at all for determining how long the email was open for.

One option that came to mind for checking the open time of an email was long polling.  Long polling (in this case) would use Javascript to contact the server every X seconds after the email was loaded.  Using those requests, it’d be trivial to find out how long it was open for.  Unfortunately, most (if not all) email clients don’t allow the execution of Javascript within emails, so that idea was completely sank.  The only option I had left was to use the beacon image to somehow determine open time.

The only option I could think of for using the image beacon without any Javascript was to redirect the image back to itself.  After much trial and error, I came up with the following.

//Open the file, and send to user.
$fileName = "../img/beacon.gif";
$fp = fopen($fileName, "r");
header("Content-type: image/gif");
while(!feof($fp)) {
    //Do a redirect for the timing.?
    sleep(2);
    if(isset($_GET['clientID'])) {
    	$redirect = $_SERVER['REQUEST_URI'];
    } else {
    	$redirect = $_SERVER['REQUEST_URI'] . "?clientID=" . $clientID;
    }
    header("Location: $redirect");
}

So what’s happening in this code?  First of all, we’re opening a small GIF file that we’re going to pretend to send to the user.  The second step is to send a header for an image file to the user so that their mail client expects one to be delivered.  This step is important because if the header isn’t sent, the browser/mail client will close the connection.  After that, you make the request sleep for a few seconds (as few or as many as you want depending on how granular you want your timing data to be) and then redirect back to the same page.  The “if” statement within the while loop is there so you can identify incoming requests and log the data accordingly.

So there you have it.  If you’ve ever wondered how people track the open time of an email, it’s probably a method very similar to this.  The only caveat to this method is that it relies on the user loading images in an email.  However, if you have a large enough sample you can just take the average open time from the users that did open it and be fairly confident with that.

 

If you liked this article, then you may like:

  • PHP Dark Arts: Multi-Processing (Part 1)
  • PHP Dark Arts: Multi-Processing (Part 2)
  • PHP Dark Arts: Shared Memory Segments (IPC)
  • PHP Dark Arts: Semaphores
  • PHP Dark Arts: GUI Programming with GTK
  • PHP Dark Arts: Sockets
  • PHP Dark Arts: Daemonizing a Process

    • CoffeeScript with WebSQL and jQuery

    Posted on by
    1

    Lately I’ve developed a distaste for Javascript.  I like what Javascript has done for the web, but I hate the syntax.  I hate that there are little “Gotch Ya!”‘s all over the language.  I don’t have to worry about that too much anymore though since I’ve started using CoffeeScript.  If you interested in learning more about it, check out the official web site, and then my “Getting Started” guide.

    Now that I’ve had a chance to dive in to CoffeeScript a bit more, I’ve started to integrate what I’m doing with other features and libraries to see what I can create.  For work I’ve been using a lot of WebSQL and jQuery, so that was the first place I took my new found CoffeeScript powers to.

    Using jQuery with CoffeeScript is really, really, easy.  For instance, let’s say we want to Ajax a page into an array:

    $.get("mypage.html", (data) ->
    myArray.push data
)

    The syntax changes just a hair, but overall it looks a lot cleaner.

    As mentioned previously, the other thing I’ve been doing a lot at work recently was working with WebSQL.  It’s been dropped by the W3C, but Webkit has implemented it already so it’s here to stay.  Anyways, CoffeeScript makes WebSQL a little more palatable.

    #Start a transaction
db.transaction( (tx) ->
    query = "SELECT * FROM table"
    tx.executeSql(query, [], (tx, results) ->
        rLength = results.rows.length
        for(i=0; i < rLength; i++)
            alert results.rows.item(i).someColName
    )
)

    With WebSQL and CoffeeScript, the syntax doesn’t change a ton, but I like the look of it better without the braces.

    Another feature that I wanted to share with out about CoffeeScript is it’s equivalent to PHP’s key, value loop construct.  In PHP, it would look like:

    foreach($key => $value as $myArray) {
    //do stuff
}

    In CoffeeScript, the equivalent is:

    for own key, value of myArray
    #Do stuff.

    The benefit of the CoffeeScript version is that it can loop over object properties as well, not just arrays.  If you have any other cool features or examples of CoffeeScript usage, drop a link (or example) into the comments.

    My History of Failure

    Posted on by
    6

    “I’ve failed over and over and over again in my life, and that is why I succeed.”

    This is my third attempt.  I’ve tried twice before and failed, but this time could be different.  It could end up in failure just like the others, but I don’t think it will.  I’m smarter this time around.  I’ve learned from my mistakes, am better at what I do, and I have more to lose.

    Encode4Free

    The first time around was in 2007.  I had just learned PHP and thought I had what it took to make a video encoding startup.  This was before the term “cloud” was popular, and my idea was that users would upload videos to my service, encode them into whatever format they liked, and then re-download them.  For revenue, I thought that a free service could be done by adding advertising to the beginning of the encoded video.  Or, a user could pay small fee to get moved to the front of the queue and have no advertisements.

    The idea seemed good, but I wasn’t dedicated to it.  I got a design going and had the back-end encoding stuff working, but I just couldn’t act on it.  Some other things went wrong too.

    • No motivation – I was still in college, didn’t need the money, and enjoyed my life just where it was.  There wasn’t really a huge motivation for me to do this.
    • No passion – I honestly didn’t care about video encoding.  Sure it’s cool, but only cool from a technical standpoint.  It wasn’t something I could really get behind.
    • Knowledge – I really had no idea how deep the programming rabbit hole went at this point.  Not knowing what good design was really put a cramp in how fast I could develop too.  In short, I was inexperienced and it showed.
    • Flawed business model – In retrospect, this should have been a freemium model.  10 minute video for free, anything over is charged.  That way people don’t get annoying advertisements added to their videos.

    Should I Get The Book?

    In 2008 and early 2009 I worked on “Should I Get The Book?”(SIGTB).  As a grad student reflecting on my undergraduate years, I realized that I spent a ridiculous amount of money on books that I never needed.  After talking to friends and classmates, it seemed this was a universal problem, and my startup idea was born.  This time, I did things right (sort of).  I took a few hours every week over the the summer and started hacking away at it.  By the end of the summer, I had a functioning product.  I did a launch, and the waited.  But nobody came.

    So what happened?  Was it competition?  Was it a poor idea?  Well, it was a lot of things.

    • Design – This is huge.  My site design sucked.  I basically ripped off a WordPress template that somebody made because it looked cool.  It had nothing to do with my idea, and it showed.  People coming to the page would say “So what is this about?”.
    • UI – The interface and experience was awful.  None of it made sense, and the changes that people did suggest I back-burnered until I “had more time”.
    • Seed Data – The idea that users leave reviews about whether they needed the book or not was great, except that nobody would do it.  Without having some reviews already in there, I found that people weren’t motivated to leave a review because they thought the site wasn’t being used.  I needed to have data seeded, but I couldn’t think of a good way to do this without paying any money (I had none at the time).
    • Focus – I tried to focus on too many schools at once with the first launch.  I should have started with one to see if the concept would stick.
    • Advertising – On a college campus, flyers and word of mouth is key.  I had zero flyers and nobody talking about me.
    • Karma – Users like getting a reward for doing something.  It doesn’t have to be money, karma works just as well.  Users get to scratch their competitive urge by posting reviews, getting karma, and checking to see if they have more then their friends. I should have done this, but didn’t.
    • Evergreen data – One of the big problems with this data is that the instructor can change books, change styles, or even stop teaching.  The data can become useless quickly.

    Once I realized most of these things I vowed to start a re-design with all of these issues addressed.  I was about 25% of the way through the mock-up when I was informed that RateMyProfessor.com had a similar feature.  It was easy to use, and they already had a huge user base.  My idea was basically sunk.

    While SIGTB flopped, it was important step for me.  I had pushed a project through to completion, learned a lot about marketing, and learned that I really need a co-founder to talk me out of bad ideas.  Most importantly though, I proved to myself that I could accomplish something.  I might have given up too quickly, but I think that the idea wouldn’t have worked well anyways.

    CampaignAlytics (Working Name)

    I started this idea out in late October of this year.  The idea is that email campaign analytics can be done better.  Currently analytics is rolled in as a feature for big mailers (Mail Chimp, etc), but it doesn’t provide the in-depth analytics that a lot of marketing folks are looking for.  We’re looking to change that.  I’ve been working for an hour a day since November 1st, and have a lot to show for it.  I pitched the idea to a local pitch night, and the reception was great.  We have a lot of great features, and do it all in a non-obtrusive way that is compatible with mailing services, or can be rolled into your custom solution using our API.

    I’m taking a different path with this startup, based on the things I’ve learned on the last two.

    • Pay for a design – The first thing I did for this web app was pay for a design.  I got a great deal, and now have a design that I can be proud of and shows off the product like nothing I could have ever come up with.
    • Co-Founder – Having someone to talk you out of bad ideas, and give you a sense of perspective is amazing.  Having someone to discuss ideas with and share the burden is pretty great too.
    • Funding – I’ve come to the conclusion that without a little bit of funding, this startup will take forever to complete.  We’re applying to the seed incubator Momentum and hopefully we’ll get funded.
    • Iterate – I’ve already gone through two different layouts and menu structures for the account/analytics page.  The second version is far better than the first because I asked for feedback from a few trusted people and implemented the changes right away.

    The thing I hope you can take away from this is that you are going to fail many times before you succeed.  Being persistent (and maybe a bit stubborn) will take you 70% of the way there, and the rest is on how good your idea is.

    PHP Bitwise Operations

    Posted on by
    9

    A bitwise operation is an operation that works on the individual bits of a number.  Yes, that means the binary(base 2) representation of a number.  These bitwise operations work by exploiting properties of binary numbers to their advantage.  For instance, if the binary representation of a number ends with a 1, it’s odd.

    Base 2 Refresher

    For those of us who don’t manipulate bits every day (including myself), I thought that a quick refresher on binary representation would be a good idea.  First, let’s create a table.

    4 3 2 1 0
    0 0 0 0 1

    Each column in the table represents a power of 2.  The first column (from the right) represents 20, the second 21, and so on.  Notice that I have a 1 in the row below the 0 column.  That stands for 20, and 20 is equal to 1.  Therefore, the number represented above (00001) is equal to 1.  On to the next table.

    4 3 2 1 0
    0 0 1 1 1
    Now the base-2 number in the bottom row is 00111.  This is equal to 22 + 21 + 20, or 4 + 2 + 1, which is equal to 7.  As you can see, using binary representation is pretty easy.  On to our final example.
    4 3 2 1 0
    1 1 0 0 1
    This example a bit (har, har) trickier.  The base-2 number in the bottom row is 11001, which is  24 + 23 +20(16 + 8 + 1) = 25.  Notice if there are zeros in the columns, we just ignore them completely.
    And that’s it!  Base-2 representation of numbers is easy, and critical for every programmer to understand.  Now on to some tricks.

    Trick 1 : Odd / Even

    One of the most useful bitwise tricks for web developers is for determining if a number is even or odd.

    for($i = 0; $i < 10; $i++) {
     if($i & 1) {
          echo "This is odd.";
     } else {
          echo "This is even.";
     }
}
    So how does help your web development?  Alternating rows.  If you deal with data in a tabular format, you nearly always need to alternate row color on a table.  Using the bitwise “AND” operator is a good way to accomplish this.

    Trick 2 : Multiplication and Division

    Using the left-shift (<<) and right-shift (>>) operators, we can easily divide and multiple by powers of two.  These operators aren’t limited to powers of two, but it makes things easier to understand.
    //Example 1
$x = 2;
echo $x &lt;&lt; 1; //4
echo $x &lt;&lt; 2; //8
 
//Example 2
$x = 16;
echo $x &gt;&gt; 1; //8
echo $x &gt;&gt; 2; //4

    In example 1, we left shift the number 2 one place.  By left shifting 1 place, it’s the same as multiplying that number by 2.  By left shifting 2 places, it’s the same multiplying the number by 2 twice.

    Example 2 is doing bitwise division.  Right shifting by 1 place is the same as dividing the number by 2.  Right shifting by 2 places, is the same is the same as dividing the number by 2 twice.

    Others

    There are most definitely a ton of other uses for bitwise operators, however these are the ones I use the most.  If you happen to have a favorite, please let everyone know about it in the comments.

    For a more in depth introduction to bitwise operations please check out http://en.wikipedia.org/wiki/Bitwise_operation and http://www.litfuel.net/tutorials/bitwise.htm

    PHP Dark Arts: Daemonizing a Process

    Posted on by
    13

    Note:  Full source code for the example can be downloaded here.

    One of the many things you don’t often do with PHP (actually, I’m not sure you do this much with any language) is daemonize a process.  A daemon is program that runs in the background (read more here).  On Unix systems, processes are usually created by forking the init process and then manipulating the process to your liking.  To create a daemon though, you need to get the init process to adopt your process.  To do that, as soon as you fork the parent process, you kill the parent.  Since you child process is parent-less, the init process generally adopts it.  Once that happens, your process has been daemonized.

    What You Need To Know

    In order to follow the example, you’ll probably want to read up on multi-processing in PHP and using POSIX in PHP.  Aside from that, keep an open mind.  There are probably better ways to do this (Nanoserv), but I think that doing it manually is a great way to learn more about systems programming and PHP.

    Step 1:  Fork It

    The first thing that you need to do when daemonizing a process in PHP is fork the process.  After that, we promptly kill the parent process so that the child process can be adopted.

    //Set the ticks
declare(ticks = 1);
 
//Fork the current process
$processID = pcntl_fork();
 
//Check to make sure the forked ok.
if ( $processID == -1 ) {
	echo "\n Error:  The process failed to fork. \n";
} else if ( $processID ) {
	//This is the parent process.
	exit;
} else {
	//We're now in the child process.
}

    Step 2:  Detach It

    Now that we have successfully forked the process and killed the parent, we need to detach the process from the terminal window.  We do this so that when the terminal window closes, our process doesn’t close with it. Once that’s done, we get our processes’ id.

    //Now, we detach from the terminal window, so that we stay alive when
//it is closed.
if ( posix_setsid() == -1 ) {
	echo "\n Error: Unable to detach from the terminal window. \n";
}
 
//Get out process id now that we've detached from the window.
$posixProcessID = posix_getpid();

    Step 3:  /var/run

    Now that we have our processes’ id, we need to let the system know about it.  To do this, we create a file in /var/run.  This file can be named anything you want, just make sure that it’s unique and it ends with the .pid extension.  In that file, we place the pid of our process and that’s it.

    //Create a new file with the process id in it.
$filePointer = fopen( "/var/run/phpprocess.pid" , "w" );
fwrite( $filePointer , $posixProcessID );
fclose( $filePointer );

    Step 4:  Do Something

    Now that all the hard stuff is done, you can get to work.  What do you want your process to do?  For this example, I have mine adding 1 + 1 every 10 seconds.  Nothing too difficult, but you can make your process do whatever you like.  For instance, you could set up a server of some sort.  Note that this the code is sitting in an infinite while loop.  This is done so that the process doesn’t exit naturally.

    //Now, do something forever.
while (true) {
	$x = 1 + 1;
	sleep(10);
}

    Step 5:  Run It

    To run this process as a daemon, all you need to do is save you file and run php <myfile>.php.  You may need to execute it as a super user depending on how you permissions are set up.  Once it’s run, you can check out the results of your hard work by running ps aux | less.  Scroll to the bottom and your process should be there.  In the screen shot below, mine is 3rd from the bottom.

    Daemonizing a ProcessNote:  Full source code for the example can be downloaded here.

    Did you like this article?  Check out the rest of the Dark Arts.

    Document Your Code

    Posted on by
    9

    When you started learning to program, documenting your code was the last thing on your mind. You’re were here to learn, and learn is what you did. As your programs started to grow in complexity, your college professors preached to you “You must document your code! What if you have to come back to it in 6 months? Will you remember how everything works?”. So, you dutifully documented your code. But then something strange happened: you got a job. In the real world there are deadlines, and that extra minute you should have taken to document the code is gone. But the thing is, you MUST document your code. If not for you, do it for me. Because guess what, I’m the one that has to go in and fix it, and knowing what’s going on would make my life a lot easier.

    A Story

    When I started my current job, one of the other programmers was assigned to get me up to speed with our CMS. My first question was “Is there any documentation I can go over?”, to which he responded “Nah, just dive in and you’ll be fine.”. So I dove in and it only got worse from there. As I looked through the code, nothing was documented. Nothing! I spent the next 3 months sifting through code trying to figure out how it all works. I eventually did figure it all out, but it was 3 months where I was working at maybe 50%-60% of my maximum efficiency. Having any sort of documentation would have made this process much easier. Not only that, but because of the lack of documentation we constantly create duplicate functionality. After all, how are you supposed to know what functionality exists if there no documentation on it?

    Now that I’ve been at my current job awhile, I’ve decided to change things. I’ve started taking 15 to 30 minutes a day to document our CMS using the PHPDoc format. It may or may not catch on with the rest of the programmers, but I know that every function and class that I write has full documentation, and it feels good when the documentation gets generated with PHPDoc. I believe the quote is “You need to be the change you believe in”.

    How to Document Your Code

    It’s easy. Really easy actually.

    /**
* Function short description
*
* This is a long description of the function.  Discuss what this function does in
* any detail that is necessary.
*
* @global void $globalVariable
* @param int $aParameter This is the description of the parameter.
* @param mixed $bParameter This is the description of another parameter.
* @return bool This is a description of the return variable.
* @todo Does something need to be done?  Put it here.
*/
function my_function($aParameter, $bParameter) {
     global $globalVariable;
 
     //If you are doing something extra complicated, make a comment inside the
     //function.  Otherwise, you code should speak for itself.
     for($i=0; $i &lt; 20; $i++) {
          echo $i;
     }
 
     return true;
}

    That’s it. That’s all you need to get started documenting PHP code. There’s a lot more you could be doing, but it’s not all that important. So long as you document the parameters, the return type, and write a description about what the function does, you have the important bits.

    So next time you think to yourself “I’ll document it later.”, just take the time to do it now. Because guess what? You aren’t going to go back. Something else is going to come up, and then the rest of us will suffer.

    PHP: Echo Versus Print(f)

    Posted on by
    4

    One of the things I remember about when I started programming in PHP was the confusion I ran in to when deciding how to print text.  Should I use the print construct?  Or maybe I should use the echo construct?  Perhaps the C-style printf function instead?  Eventually most people just latch on to the echo construct because “That’s what everyone else is doing”.  Is that the fastest method to use though? Well, that’s what I’m going to find out.

    The Test

    The test I decided to use loops 1000 times, and inside the main loop, there is a sub-loop creating the output which runs from 0 to N (where N is the value of the outer-loop counter).

    1
2
3
4
5
6
7

    $start = microtime(true);
for($i = 0; $i < 1000; $i++) {
     for($x = 0; $x < $i; $x++) {
          //Output here.
     }
}
$end = microtime(true);

    After the test has been completed, I just wrote the times to a text file for collection later.

    The Results

    • echo – 16.470 seconds
    • print – 16.473 seconds
    • printf – 16.432 seconds

    As I expected, the difference between print and echo were negligible.  I suspect that they use the same underlying code.  The real surprise was from printf(), which is a function and is supposed to have taken longer because of the overhead of calling it.  I’m at a loss as to why it’s faster than the other two, but the difference isn’t really significant enough to worry about.  If someone wanted to be a bit more scientific about this, this program should be run probably a couple thousand times so that we can get a good sample and see what happens then.

    If you would like to run this code yourself and let me know your results, it’s available for download here.

    PHP Dark Arts: Shared Memory Segments (IPC)

    Posted on by
    11

    Note: The full source code for the examples can be downloaded here.

    In my previous articles on using PHP for multi-process programming, we kept it very simple.  By simple, I mean we didn’t have any inter-process communication (IPC).  IPC is a set of techniques for the exchange of data amongst separate processes and/or threads.  There are many different ways to set up IPC, such as files, signals, sockets, pipes, semaphores, shared memory, and message passing.  This time around, we’re going to cover PHP’s implementation of shared memory segments.

    So what does it mean to share a memory segment?  It means that the program will create a section of memory that can be accessed by other processes on the system.  Normally this isn’t the case, since most processes have mutually exclusive address spaces.  The benefit of using a shared memory segment is that communication between processes is extremely fast.  The main downside is that processes must be running on the same machine (and same processor in some cases), where as other types of IPC can be used over a network.

    Creating a Shared Memory Segment

    In order to share memory between processes, you first need to create the shared memory segment.  This is accomplished with PHP’s shm_attach() function.  The shm_attach() function takes 3 parameters.

    • $key (int) – This is an integer value that identifies your shared memory segment.  If you used 123456 for this value in one process, you would need to use the same key in another process to access the shared memory segment.
    • $memsize (int) -  This is the amount of memory (in bytes) you would like to share.  Deciding how large this should be is sort of a pain.  The easiest way it to simply pick an arbitrary amount and make sure anything you try to share isn’t larger then that.  The best way would be to know the maximum size of data you will need to share and set it to that value.
    • $perm (int) – The permissions for the shared memory segment.  These permissions follow typical Unix style permissions.  The default is 0666 which means that everyone can read from and write to this memory segment.

    On success, shm_attach() returns an identifier of the shared memory segment.

    1
2
3
4
5
6
7

    //Define shared memory segment properties.
$key = "987654";
$permissions = 0666;
$size = 1024;
 
//Create or open the shared memory segment.
$segment = shm_attach($key, $size, $permissions);

    Using Your Shared Memory Segment

    Now that you have created a shared memory segment, you’ll obviously want to try it out.  Using it is pretty straight forward with the shm_put_var() and shm_get_var() functions.  They work as you might expect them to.  shm_put_var() has 3 parameters:

    • The shared memory segment key.
    • An integer used basically as an index on the data (making it easier to retrieve).
    • Some variable.  This can be anything so long as it’s smaller than the size you’ve defined the memory segment to be.

    The shm_get_var() function is very similar.  It only takes 2 parameters, one of which is the key for the shared memory segment, and the other is the integer value (index) associated with the stored data.  Now that you know what the functions do, let’s put it all together in a little program.

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

    <?php
//Check the command line arguments
if(sizeof($argv) < 2) {
     echo  "Usage: php shared_memory.php <send|get|delete> <integer identifier> <value>\n";
     exit;
}
 
//Define shared memory segment properties.
$key = "987654";
$permissions = 0666;
$size = 1024;
 
//Create or open the shared memory segment.
$segment = shm_attach($key, $size, $permissions);
 
//Handle operations for the segment.
switch($argv[1]) {
     case "send":
          shm_put_var($segment, $argv[2], $argv[3]);
          echo "Message sent to shared memory segment.\n";
          break;
     case "get":
          $data = shm_get_var($segment, $argv[2]);
          echo "Received data: {$data}\n";
          break;
     case "delete":
          shm_remove($segment);
          echo "Shared memory segment released.\n";
          break;
}
?>

    A quick glance at the code reveals that I’m making this program a little interactive for easier use.  All it does is check to make sure your command line arguments are correct, creates / opens the shared memory segment, and then performs the given operation on the segment.  An example:

    <->php shared_memory.php
Usage: php shared_memory.php send|get|delete integer identifier value;
<->php shared_memory.php send 1 "Hello.  This is the shared memory segment."
Message sent to shared memory segment.
<->php shared_memory.php get 1
Received data: Hello.  This is the shared memory segment.
<->php shared_memory.php delete
Shared memory segment content deleted.

    You’ll also notice that at the end I delete the contents of the shared memory segment.  It’s important to understand the difference between detaching and deleting(removing) here.  When you detach the shared memory segment, it’s possible for your data to still exist in memory until it’s overwritten by something else.  This poses a big security threat, so you should always delete(remove) the data first.  So how do we release the memory?  By using the shm_detach() function.

    <->
    //Define shared memory segment properties.
    $key = “987654″;
    $permissions = 0666;
    $size = 1024;

    //Create or open the shared memory segment.
    $segment = shm_attach($key, $size, $permissions);

    //Detach the memory segment.
    shm_detach($segment);

    Download

    Now that your done, check out the full source code for the examples here.  In the near future I hope to have a virtual machine available for download, so users who don't want to mess with configuration can just start it up and go.