Ed's World

After following one of my two previous blog posts, you should be in a state ready to pub/sub to/from your web pages. Here’s a tiny bit of code that shows how you can push messages:

Thread t = new Thread(){
	int count = 0;
	@Override
	public void run() {
		while (true) {
			Mutable msg = bayeux.newMessage();
			msg.setChannel("/test");
			HashMap map = new HashMap();
			map.put("key1", count + "");
			msg.setData(map);

			ServerChannel c = bayeux.getChannel("/test");
			if (c != null) {
				c.publish(null, msg);
			}
			count++;
			try {
				Thread.sleep(5000);
			} catch (InterruptedException e) {
				e.printStackTrace();
			}
		}
	}
};
t.start();

I put that in the init() call of BayeuxInitialiser. Now when you subscribe to /test, you will get a message with an ever increasing value.

Very easy, very powerful. Would make a nice bridge between MQTT and the web, especially as it doesn’t rely on websockets etc.

How many keywords can I stuff into that title!! I succeeded to get Jetty, CometD and Dojo working together, without OSGi in the last blog post. However, moving that into an OSGi environment was sadly not as easy as it should have been, but I got there! This post should help you get things up and running – there are bits I understand, and bits I don’t!! It is as cut down as I could make it. Here goes:

In eclipse, make yourself a new Plugin Project, mine’s called com.jellard.comet, choosing the Equinox OSGi framework. Edit MANIFEST.MF to look like below:

Add this to the MANIFEST.MF as well:
Web-ContextPath: /comettest

Put these files in your project, and add them to the build path: cometd-java-server-2.3.1.jar, bayeux-api-2.3.1.jar, cometd-java-common-2.3.1.jar

Copy this file into a new class, com.jellard.comet.BayeuxInitialiser:

package com.jellard.comet;

import java.io.IOException;

import javax.servlet.GenericServlet;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;

import org.cometd.bayeux.server.BayeuxServer;
import org.cometd.bayeux.server.ServerChannel;
import org.cometd.bayeux.server.ServerMessage;
import org.cometd.bayeux.server.ServerSession;
import org.cometd.server.DefaultSecurityPolicy;

public class BayeuxInitialiser extends GenericServlet
{
	private static final long serialVersionUID = 4696913071422149884L;

	public void init() throws ServletException
    {
        BayeuxServer bayeux = (BayeuxServer)getServletContext().getAttribute(BayeuxServer.ATTRIBUTE);
        bayeux.setSecurityPolicy(new DefaultSecurityPolicy(){

			@Override
			public boolean canCreate(BayeuxServer server,
					ServerSession session, String channelId,
					ServerMessage message) {
				return true;
			}

			@Override
			public boolean canHandshake(BayeuxServer server,
					ServerSession session, ServerMessage message) {
				return true;
			}

			@Override
			public boolean canPublish(BayeuxServer server,
					ServerSession session, ServerChannel channel,
					ServerMessage message) {
				return true;
			}

			@Override
			public boolean canSubscribe(BayeuxServer server,
					ServerSession session, ServerChannel channel,
					ServerMessage message) {
				return true;
			}

        });
        new HelloService(bayeux);
    }

    public void service(ServletRequest request, ServletResponse response) throws ServletException, IOException
    {
        throw new ServletException();
    }
}

Make a new class, com.jellard.comet.HelloService, and paste this in:

package com.jellard.comet;

import java.util.HashMap;
import java.util.Map;

import org.cometd.bayeux.Message;
import org.cometd.bayeux.server.BayeuxServer;
import org.cometd.bayeux.server.ServerSession;
import org.cometd.server.AbstractService;

public class HelloService extends AbstractService
{
    public HelloService(BayeuxServer bayeux)
    {
        super(bayeux, "hello");
        addService("/service/hello", "processHello");
    }

    public void processHello(final ServerSession remote, Message message)
    {
        Map input = message.getDataAsMap();
        String name = (String)input.get("name");

        Map output = new HashMap();
        output.put("greeting", "Hello, " + name);
        remote.deliver(getServerSession(), "/hello", output, null);

        Thread t = new Thread(){
        	int count = 0;
			@Override
			public void run() {
				while (true) {
					remote.deliver(getServerSession(), "/reply", count + "", null);
					System.out.println(count);
					count++;
					try {
						Thread.sleep(5000);
					} catch (InterruptedException e) {
						// TODO Auto-generated catch block
						e.printStackTrace();
					}
				}
			}

        };
        t.start();
    }
}

Create folders ~/jetty/etc/ and place this file in, as jetty.xml:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">

<!-- =============================================================== -->
<!-- Configure the Jetty Server                                      -->
<!--                                                                 -->
<!-- Documentation of this file format can be found at:              -->
<!-- http://docs.codehaus.org/display/JETTY/jetty.xml                -->
<!--                                                                 -->
<!-- =============================================================== -->

<Configure id="Server" class="org.eclipse.jetty.server.Server">

    <!-- =========================================================== -->
    <!-- Set connectors                                              -->
    <!-- =========================================================== -->

    <Call name="addConnector">
      <Arg>
          <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
            <Set name="port"><SystemProperty name="jetty.port" default="8080"/></Set>
          </New>
      </Arg>
    </Call>

    <!-- =========================================================== -->
    <!-- Set handler Collection Structure                            -->
    <!-- =========================================================== -->

    <Set name="handler">
      <New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
        <Set name="handlers">
         <Array type="org.eclipse.jetty.server.Handler">
           <Item>
				<New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection">

			</New>
           </Item>
           <Item>
             <New id="DefaultHandler" class="org.eclipse.jetty.server.handler.DefaultHandler"/>
           </Item>
           <Item>
             <New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler"/>
           </Item>
         </Array>
        </Set>
      </New>
    </Set>
    <!-- =========================================================== -->
    <!-- Configure the deployment manager                            -->
    <!--                                                             -->
    <!-- Sets up 2 monitored dir app providers that are configured   -->
    <!-- to behave in a similaraly to the legacy ContextDeployer     -->
    <!-- and WebAppDeployer from previous versions of Jetty.         -->
    <!-- =========================================================== -->
    <Call name="addBean">
      <Arg>
        <New id="DeploymentManager" class="org.eclipse.jetty.deploy.DeploymentManager">
          <Set name="contexts">
            <Ref id="Contexts" />
          </Set>
          <Call name="setContextAttribute">
            <Arg>org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern</Arg>
            <Arg>.*/jsp-api-[^/]*\.jar$|.*/jsp-[^/]*\.jar$</Arg>
          </Call>
          <!-- Providers of OSGi Apps -->
          <Call name="addAppProvider">
            <Arg>
              <New class="org.eclipse.jetty.osgi.boot.OSGiAppProvider">
              <!--
                <Set name="defaultsDescriptor"><Property name="jetty.home" default="."/>/etc/webdefault.xml</Set>
              -->
                <Set name="scanInterval">5</Set>
                <Set name="contextXmlDir"><Property name="jetty.home" default="." />/contexts</Set>
                <!-- comma separated list of bundle symbolic names that
                    contain custom tag libraries (*.tld files)
                    if those bundles don't exist or can't be loaded no errors or warning will be issued!
                    this default value is to plug the tld files of the reference implementation of JSF -->
                <Set name="tldBundles"><Property name="org.eclipse.jetty.osgi.tldsbundles"
                     default="javax.faces.jsf-impl" /></Set>
              </New>
            </Arg>
          </Call>

        </New>
      </Arg>
    </Call>

    <!-- =========================================================== -->
    <!-- extra options                                               -->
    <!-- =========================================================== -->
    <Set name="stopAtShutdown">true</Set>
    <Set name="sendServerVersion">true</Set>
    <Set name="sendDateHeader">true</Set>
    <Set name="gracefulShutdown">1000</Set>

</Configure>

Create a XML file, WEB-INF/web.xml, in your project:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
	version="2.5">
	<servlet>
		<servlet-name>cometd</servlet-name>
		<servlet-class>org.cometd.server.CometdServlet</servlet-class>
		<load-on-startup>1</load-on-startup>
	</servlet>
	<servlet-mapping>
		<servlet-name>cometd</servlet-name>
		<url-pattern>/cometd/*</url-pattern>
	</servlet-mapping>
	<servlet>
		<servlet-name>initializer</servlet-name>
		<servlet-class>com.jellard.comet.BayeuxInitialiser</servlet-class>
		<load-on-startup>2</load-on-startup>
	</servlet>
	<filter>
		<filter-name>cross-origin</filter-name>
		<filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class>
	</filter>
	<filter-mapping>
		<filter-name>cross-origin</filter-name>
		<url-pattern>/cometd/*</url-pattern>
	</filter-mapping>
</web-app>

Create a html folder in the root of your project. In there, create a js/dojo folder, and extract a build of dojo into there (so you will have a html/dojo/dojox folder, for example).

Also in the html folder, make a file, comet.html, and paste in this:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
    <script type="text/javascript" src="js/dojo/dojo/dojo.js"></script>
    <script type="text/javascript">
        dojo.require("dojox.cometd");
        dojo.addOnLoad(function() {
            // Disconnect when the page unloads
            dojo.addOnUnload(function() {
               dojox.cometd.disconnect(true);
            });

            var cometURL = "/comettest/cometd";
            dojox.cometd.init(cometURL);
        });
        function subscribe() {
        	dojox.cometd.subscribe("/*", console, "log");
        }
        function publish() {
        	dojox.cometd.publish('/service/hello', { name: 'World' })
        }
    </script>
</head>
<body>
    <input type="button" onclick="subscribe()" value="1. Subscribe" />
    <input type="button" onclick="publish();" value="2. Publish" />
</body>
</html>

Right-click your project, Run, Run As… and make a new configuration under OSGi Framework. Select the following bundles, and anything else they require (I ended up with 55 selected!)

In the arguments tab, you need to add this to your VM args:
-Djetty.home=/Users/ed/jetty (if your home dir is /Users/ed)

Run the OSGi framework, and point your browser to http://localhost:8080/comettest/html/comet.html

Open up developer tools/firefox, and you should see a pending request. Click the subscribe and publish button, and as if by magic, every 5 seconds, you should see an object appear in your console.

There’s not a great deal of up to date documentation of how to get dojo and cometd working with the Bayeux protocol. This quick blog post should get you up and running, and is based off the “Primer” sample code that you can get here (but you don’t need it for this): http://cometd.org/documentation/howtos/primer

The following code will show you how to set everything up and then publish a message from the webpage. This will kick off a thread in Jetty that publishes an ever-increasing number every 5 seconds that will be shown on screen.

Prereqs:
Jetty 8.x server runtime setup and installed in eclipse (well documented elsewhere)
Dojo 1.7

Create a Dynamic Web Project in eclipse, pointing it at Jetty.

Add to your build path:
bayeux-api-2.3.1.jar
cometd-java-common-2.3.1.jar
cometd-java-server-2.3.1.jar

Create a GenericServlet, calling it test.BayeuxInitialiser, paste in this:

/*
 * Copyright (c) 2010 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package test;

import java.io.IOException;

import javax.servlet.GenericServlet;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;

import org.cometd.bayeux.server.BayeuxServer;
import org.cometd.bayeux.server.ServerChannel;
import org.cometd.bayeux.server.ServerMessage;
import org.cometd.bayeux.server.ServerSession;
import org.cometd.server.DefaultSecurityPolicy;

public class BayeuxInitialiser extends GenericServlet
{
	private static final long serialVersionUID = -9089442901563633963L;

	public void init() throws ServletException
    {
        BayeuxServer bayeux = (BayeuxServer)getServletContext().getAttribute(BayeuxServer.ATTRIBUTE);
        bayeux.setSecurityPolicy(new DefaultSecurityPolicy(){

			@Override
			public boolean canCreate(BayeuxServer server,
					ServerSession session, String channelId,
					ServerMessage message) {
				return true;
			}

			@Override
			public boolean canHandshake(BayeuxServer server,
					ServerSession session, ServerMessage message) {
				return true;
			}

			@Override
			public boolean canPublish(BayeuxServer server,
					ServerSession session, ServerChannel channel,
					ServerMessage message) {
				return true;
			}

			@Override
			public boolean canSubscribe(BayeuxServer server,
					ServerSession session, ServerChannel channel,
					ServerMessage message) {
				return true;
			}

        });
        new HelloService(bayeux);
    }

    public void service(ServletRequest request, ServletResponse response) throws ServletException, IOException
    {
        throw new ServletException();
    }
}

Create a test.HelloService class, and paste in this:

/*
 * Copyright (c) 2010 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package test;

import java.util.Map;
import java.util.HashMap;

import org.cometd.bayeux.Message;
import org.cometd.bayeux.server.BayeuxServer;
import org.cometd.bayeux.server.ServerSession;
import org.cometd.server.AbstractService;

public class HelloService extends AbstractService
{
    public HelloService(BayeuxServer bayeux)
    {
        super(bayeux, "hello");
        addService("/service/hello", "processHello");
        System.out.println("New HelloService");
    }

    public void processHello(final ServerSession remote, Message message)
    {
        Map input = message.getDataAsMap();
        String name = (String)input.get("name");
        Map output = new HashMap();
        output.put("greeting", "Hello, " + name);
        remote.deliver(getServerSession(), "/hello", output, null);

        Thread t = new Thread(){
        	int count = 0;
			@Override
			public void run() {
				while (true) {
					remote.deliver(getServerSession(), "/reply", count + "", null);
					System.out.println(count);
					count++;
					try {
						Thread.sleep(5000);
					} catch (InterruptedException e) {
						// TODO Auto-generated catch block
						e.printStackTrace();
					}
				}
			}

        };
        t.start();
    }
}

Now, in your WebContent directory:

Unzip dojo into js/dojo (so you will have js/dojo/dojo/* js/dojo/dijit/* etc.)

Create index.jsp:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
    <script type="text/javascript" src="js/dojo/dojo/dojo.js"></script>
    <script type="text/javascript">
        dojo.require("dojox.cometd");
        dojo.addOnLoad(function() {
            // Disconnect when the page unloads
            dojo.addOnUnload(function() {
               dojox.cometd.disconnect(true);
            });

            var cometURL = "cometd";
            dojox.cometd.init(cometURL);
        });
        function subscribe() {
        	dojox.cometd.subscribe("/*", console, "log");
        }
        function publish() {
        	dojox.cometd.publish('/service/hello', { name: 'World' })
        }
    </script>
</head>
<body>
    <input type="button" onclick="subscribe()" value="1. Subscribe" />
    <input type="button" onclick="publish();" value="2. Publish" />
</body>
</html>

Replace web.xml with this:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">

    <servlet>
        <servlet-name>cometd</servlet-name>
        <servlet-class>org.cometd.server.CometdServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>

        <servlet-name>cometd</servlet-name>
        <url-pattern>/cometd/*</url-pattern>
    </servlet-mapping>

    <servlet>
        <servlet-name>initializer</servlet-name>
        <servlet-class>test.BayeuxInitialiser</servlet-class>

        <load-on-startup>2</load-on-startup>
    </servlet>

    <filter>
        <filter-name>cross-origin</filter-name>
        <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class>
    </filter>
    <filter-mapping>

        <filter-name>cross-origin</filter-name>
        <url-pattern>/cometd/*</url-pattern>
    </filter-mapping>

</web-app>

Now point your browser to index.jsp, click the subscribe button, and click the publish button. You should see messages appear on the screen and in your console.

Hope this helps someone!

I want to check every AJAX request for a “is logged out” flag. I also want to do something nice for every AJAX error, globally (in this case, I am just going to make a dialog showing the stacktrace from tomcat).

I could do something in every load: and error:, but this does it globally:

Error handling (assumes something is subscribed to the errors topic that displays a dialog, or whatever):

(function(){
   var oldxhr = dojo.xhr;
   dojo.xhr = function(){
      return oldxhr.apply(dojo, arguments).addErrback(function(e, v){
      dojo.publish("errors", [{text:e.responseText}]);
      });
   };
})();


Checking all JSON (assumes something has subscribed to the checklogin topic to do the logic):

dojo.contentHandlers.json = function(xhr) {
   var json = dojo.fromJson(xhr.responseText || null);
   dojo.publish("checklogin", [{j:json}]);
   return json;
}


In the move to put as much of my house onto twitter as possible, MythTV was the next victim. I’d previously made a PHP script that sent MSN alerts, but now I don’t use MSN, it needed a revamp.

I knocked up a small PHP script that grabs the backend status, finds what is being recorded or watched, compares with the previous run, and sends a message to twitter using wget.

To use it, grab the script, change the username/password variables, chmod +x it, and then add it to your “Startup Applications” or equivalent if you’re not using Ubuntu.

I’ve recently created a new website for my scout group, Highfield Scouts, and have put up a google calendar. More out of a challenge than a real need, I’ve knocked up a script that uses the Google Data API for PHP to query my calendar feed, grab the next event, and make a fake handwritten postit note. The script runs every hour to keep the images up to date.

I’ve made a little markup system so that the whole postit can be a link, and the time and location can be overridden (child protection malarky – can’t just put a time and location for some things!)

It’s all written in PHP, using the GD graphics features to put the text on a background image. If you’re interested in the code, comment, and I’ll give it to you (if you link to me!)

If you have HTML/javascript files sitting on your harddisk and you want to open them in firefox and actually use the javascript, you need to change one simple setting.

First, go to about:config (just type it in the address bar and hit enter)

Then filter for “origin”.

Change “security.fileuri.strict_origin_policy” to false.

You should now be able to use local javascript files.

Since writing “Get number of unread posts from google mail”, I realised that I didn’t care about a lot of emails that I get on a regular basis, things like ebuyer emails, stock-market alerts etc. I only want to be notified when I’ve got emails from real people.

By applying a “Newsletters” label on GMail to all automated messages by filtering messages, I can now see how many unread emails I have, then subtract the number of unread newsletters I have, and flash the ambient light that number of times.

Grab getGoogleMail.sh, change the usernames and passwords, and change “Newsletters” to whatever your google label is. Then use the $count variable to do whatever you want.

Something that drives me nuts is every so often there’ll be an accident on the M3 or M27 and the entire surrounding area will be flooded with people trying to avoid the deadlock, only to find they’re on a tiny road with thousands of other people. Yes, I could check the traffic websites each morning, or do some intelligent parsing of traffic RSS feeds, but the first option requires effort before 9am (not going to happen), and the second option is never going to be reliable enough – it’d be too hard to programmatically determine how bad the conditions on the road really are.

Read more…

So, after yesterday’s Pinewood Derby with funky racing traffic lights, I looked for a simple, quick photo album tool to display the photos, and stumbled on Piwigo, which looked reasonable and the whole “Simple powerful” philosophy seemed to match what I wanted. So, I downloaded the single php installer script, which automagically downloads the latest version and installs everything. So far, so good.

Now to actually create an album. Yes, pretty easy. Upload a file, not so easy, but doable. Get that file to be shown – next to impossible. This is the worlds most convoluted software I have ever seen! To get a photo to be shown, you first have to syncronise, then go and create thumbnails – but wait, by default the synchronise does a dummy run, so go back and tick a tick-box. Grr, really not usable. So, I tried looking at the documentation (which goes against my beliefs of websites should never, ever, need any help – they should be usable) and even that is appauling! It’s a wiki with a very poor navigation structure (if there is any structure), so impossible to find anything useful. Turn to the forums to find others having the same problem, and being told to read the instructions…

Right, rant over, software uninstalled.

Plogger was the next on the list, the demo looked clean and simple – just what I needed (PHP/MySQL based). Installed it in a couple of minutes, saw a test album had been created for me so uploaded a photo and job done – it was there! Then spent the next hour cropping and adjusting the photos, uploaded them, and bob’s your uncle, I have a nice and simple photo gallery for the scouts! The styling is very minimal, it’s designed to be part of a larger site, although there is one style that’s a little less minimal which I’ve gone for as there’s no “larger site” for this.

So steer clear of piwigo, and let me know if you have any recommendations…