How do I implement basic “Long Polling”?

Question 1

I can find lots of information on how Long Polling works (For example, this, and this), but no simple examples of how to implement this in code.

All I can find is cometd, which relies on the Dojo JS framework, and a fairly complex server system..

Basically, how would I use Apache to serve the requests, and how would I write a simple script (say, in PHP) which would "long-poll" the server for new messages?

The example doesn't have to be scaleable, secure or complete, it just needs to work!

Question 2

It's simpler than I initially thought.. Basically you have a page that does nothing, until the data you want to send is available (say, a new message arrives).

Here is a really basic example, which sends a simple string after 2-10 seconds. 1 in 3 chance of returning an error 404 (to show error handling in the coming Javascript example)

msgsrv.php

<?php
if(rand(1,3) == 1){
    /* Fake an error */
    header("HTTP/1.0 404 Not Found");
    die();
}

/* Send a string after a random number of seconds (2-10) */
sleep(rand(2,10));
echo("Hi! Have a random number: " . rand(1,10));
?>

Note: With a real site, running this on a regular web-server like Apache will quickly tie up all the "worker threads" and leave it unable to respond to other requests.. There are ways around this, but it is recommended to write a "long-poll server" in something like Python's twisted, which does not rely on one thread per request. cometD is an popular one (which is available in several languages), and Tornado is a new framework made specifically for such tasks (it was built for FriendFeed's long-polling code)... but as a simple example, Apache is more than adequate! This script could easily be written in any language (I chose Apache/PHP as they are very common, and I happened to be running them locally)

Then, in Javascript, you request the above file (msg_srv.php), and wait for a response. When you get one, you act upon the data. Then you request the file and wait again, act upon the data (and repeat)

What follows is an example of such a page.. When the page is loaded, it sends the initial request for the msgsrv.php file.. If it succeeds, we append the message to the #messages div, then after 1 second we call the waitForMsg function again, which triggers the wait.

The 1 second setTimeout() is a really basic rate-limiter, it works fine without this, but if msgsrv.php always returns instantly (with a syntax error, for example) - you flood the browser and it can quickly freeze up. This would better be done checking if the file contains a valid JSON response, and/or keeping a running total of requests-per-minute/second, and pausing appropriately.

If the page errors, it appends the error to the #messages div, waits 15 seconds and then tries again (identical to how we wait 1 second after each message)

The nice thing about this approach is it is very resilient. If the clients internet connection dies, it will timeout, then try and reconnect - this is inherent in how long polling works, no complicated error-handling is required

Anyway, the long_poller.htm code, using the jQuery framework:

<html>
<head>
    <title>BargePoller</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js" type="text/javascript" charset="utf-8"></script>

    <style type="text/css" media="screen">
      body{ background:#000;color:#fff;font-size:.9em; }
      .msg{ background:#aaa;padding:.2em; border-bottom:1px #000 solid}
      .old{ background-color:#246499;}
      .new{ background-color:#3B9957;}
    .error{ background-color:#992E36;}
    </style>

    <script type="text/javascript" charset="utf-8">
    function addmsg(type, msg){
        /* Simple helper to add a div.
        type is the name of a CSS class (old/new/error).
        msg is the contents of the div */
        $("#messages").append(
            "<div class='msg "+ type +"'>"+ msg +"</div>"
        );
    }

    function waitForMsg(){
        /* This requests the url "msgsrv.php"
        When it complete (or errors)*/
        $.ajax({
            type: "GET",
            url: "msgsrv.php",

            async: true, /* If set to non-async, browser shows page as "Loading.."*/
            cache: false,
            timeout:50000, /* Timeout in ms */

            success: function(data){ /* called when request to barge.php completes */
                addmsg("new", data); /* Add response to a .msg div (with the "new" class)*/
                setTimeout(
                    waitForMsg, /* Request next message */
                    1000 /* ..after 1 seconds */
                );
            },
            error: function(XMLHttpRequest, textStatus, errorThrown){
                addmsg("error", textStatus + " (" + errorThrown + ")");
                setTimeout(
                    waitForMsg, /* Try again after.. */
                    15000); /* milliseconds (15seconds) */
            }
        });
    };

    $(document).ready(function(){
        waitForMsg(); /* Start the inital request */
    });
    </script>
</head>
<body>
    <div id="messages">
        <div class="msg old">
            BargePoll message requester!
        </div>
    </div>
</body>
</html>

Question 3

You can convince PHP's curl backend to stop doing the 100-continue-thing by setting an explicit request header:

curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));

This way you can post a request however long you would ever want and curl will not do the dual phase post.

I've blogged about this nearly two years ago.

Question 4

A rough idea to start you:

<?php   
  session_start();

  if( isset( $_GET['logout'] ) )
  {
    session_destroy();
    header('Location: ../logout.php');
    exit;
  }

  if( !isset( $_SESSION['login'] ) )
  {
    if( !isset( $_SERVER['PHP_AUTH_USER'] ) || !isset( $_SERVER['PHP_AUTH_PW'] ) )
    {
      header("HTTP/1.0 401 Unauthorized");
      header("WWW-authenticate: Basic realm="Tets"");
      header("Content-type: text/html");
      // Print HTML that a password is required
      exit;
    }
    else
    {
      // Validate the $_SERVER['PHP_AUTH_USER'] & $_SERVER['PHP_AUTH_PW']
      if( $_SERVER['PHP_AUTH_USER']!='TheUsername'
          || $_SERVER['PHP_AUTH_PW']!='ThePassword' )
      {
        // Invalid: 401 Error & Exit
        header("HTTP/1.0 401 Unauthorized");
        header("WWW-authenticate: Basic realm="Tets"");
        header("Content-type: text/html");
        // Print HTML that a username or password is not valid
        exit;
      }
      else
      {
        // Valid
        $_SESSION['login']=true;
      }
    }
  }
?>
// The rest of the page is then displayed like normal

Question 5

Quantity:

Actually, I don't know acts_as_shopping cart and what exactly it do. But anyway look at the 'standard' realization in Agile Web Development with Rails (4ed), page 117. You can just steal those cart to your application, or whole the application. I think this book is about you. :)

Product properties:

It is VERY important to clarify the problem from the beginning and its potential complications due to the demands of the owner. So you do not have trouble link there.

Any 'attributes' have to be known and classified carefully. Let say, color is a simple anywhere (red is red in america, africa or europe), but the size - is not. Size of what? Cloth size depends on region dimension, culture, cloth type etc and can be just one value ("M", "S" or "46") and depends on product origin... Size of 3D is a 3 numerics (can be inches, cm, m) and depends on region of sale.

So, due to OOP thinking, you have to write every product 'attribute' in a stand-alone ActiveRecord model (which has_many product_items) but you have to know exactly HOW you will be able to search them as fast as possible.

At the finish i think you accomplish something like that.

size = Size.new(:origin => :us, :value => "m")
size.origin
#=> "us"
size.value
#=> "m"
size.with_analogs # model knows how to translate sizes between each other
#=> [<self>, <#Size @origin: "europe", @value: 46>, <#Size. @origin: "europe", @value: 45.5>, <#Size @origin: "australia", @value: ...>]

So despite the fact that the product item has (actually belongs_to) only one particular size, you can find a similar size of similar goods like that:

product = Product.find(1)
Product.where(:name => "%#{product.name}", :size => {:id => [product.size.with_analogs.map(&:id)]})

Or you can store in the database common measurement learning STI how to translate them, but it little bit complicated.

class Size
class EuropeSize < Size     # STI
class UsSize < Size         # STI
class AustraliaSize < Size  # STI

...

Question 6

The implementation details of a long poll server would vary so much from platform to platform that your assumptions might not be correct.

I implemented a COMET server for our website using .NET. I leveraged HttpListener to do all the boring http stuff and Microsoft CCR to deal with all the async IO. It uses a pool of threads to service requests as and when they come in. It's not a thread per client, but it's not single threaded either generally requiring a few tens of threads to stay fluid as user numbers rise. This approach means that we scale easily across multiple CPU cores. CCRs async enumerator pattern really helped keep the asynchronous logic nice and tidy, and I can read the code fairly easily a year later.

This approach has proved extremely scalable. I've tested up to 20000 clients, whereupon we became bound by network IO. It handles all our clients (who are "permanently" connected, reconnecting every 30s) ticking along at 1-2% server load. It's definitely worth reconsidering your assumption that you must either choose an event loop architecture as opposed to multiple threads. The middle ground works very nicely for me, and the .NET asynchronous programming model for dealing with IO bound tasks really takes you away from needing to micro-manage threads. Effectively, when there's IO data to process, a thread is borrowed from the pool to do that processing, and subsequently returned to the pool ready to service another request. All the complicated IOCP stuff is abstracted away.

How do I implement basic “Long Polling”?

Answers

php,http,comet