vWork Development

Dec 21

Have you seen our Kanban Camera? -

Our kanbancam runs 24/7 (with archives) at http://www.kanbancam.com/

This is the view of our Kanban wall. To the right of frame you can see our information radiator showing our Hudson CI status.

Oct 28

Fixing 403 Forbidden Issue: OSX + Passenger Preference Pane + Apache

I recently installed the Passenger Preferences Pane from Finger Tips. This is a great way to run a Rails app in a local development environment on a Mac.

During the install process I did nothing different or flash, just followed the instructions.

After trying to hit the URL for my local app for the first time, I got this error message:

[error] [client 127.0.0.1] (13)Permission denied: access to / denied

I followed all the Google I could find about checking the directive was correct, and also that the permissions of my project dir were 755. They were.

After some more ‘advanced’ Google, I found this gem of a comment, which points out that every level of the entire path to your project needs to have appropriate (something like 755) permissions.

E.g, if the path to your project is /usr/local/apps/foo:

chmod 755 /usr
chmod 755 /usr/local
chmod 755 /usr/local/apps
chmod 755 /usr/local/apps/foo

This fixed the problem for me, without the need to restart anything.

-Jonathan

Sep 26

Couch and Flex. First Steps.

I had this idea that I wanted to replace TraceTarget (the default Flex logging target) with CouchTarget. That way, instead of producing .csv text I could sort and filter in excel. I could get realtime (or close to) in couch and write views/shows that narrowed down my debug session.

So off I went, and it did not go well.

Step 1: Installing CouchDB

I have an Ubuntu instance so I just installed couch with Aptitude. I did some tutorials and was loving Couch in minutes.

Step 2: Talk to couch from flex.

Ouch. So couch wants ‘PUT’ methods, and fair enough too. But I know I can’t do that from URLLoader. I played with some other solutions, like the as3http library, but I really didn’t want something so OTT for a simple logger. So I searched for a magic bullet I knew had to be there.

PUT when you can’t

In RubyOnRails rest calls, you simply provide _method=put as a parameter to a POST and you get PUT. This is because some clients (Flash player included) cannot PUT, they can only GET and POST.

So I went looking throught the CouchDB docs to see if it supported an alternative to ‘PUT’. Nothing came up. I was still sure it MUST support it somehow though. So I went looking around for other alternatives. I finally stumble upon the X-HTTP-Method-Override header.

Eureka! right? wrong..

X-HTTP-Method-Override support

I tried it, it did not work. I thought, well, couch does not support this…

Searching for X-HTTP-Method-Override and CouchDB finally turned up a changelog, where it was added in some version, which was clearly newer than the 0.10 version Ubuntu installs.

Moral

Install CouchDB >1.0.

CouchOne do great installers

Example

Heres the crufty first draft of my Logger

It has dependencies you wont have so don’t expect it to run, or play nice. But it does show how I got posts going. RestfulX provided an easy method for adding headers and also kept me believing it was possible with URLLoader, because they do it too.

Aug 25

Builder pattern is so easy in Robotlegs

The Builder Pattern is a breeze in Robotlegs

public class ConcreteWidgetBuilder implements IWidgetBuilder {

  [Inject]
  public var injector:IInjector;

  public function buildPart(parameterised:IThing):IWidget {
    var widget:IWidget = injector.getInstance(IWidget);
    widget.param = parameterised;

    return widget;
  }

}

Parameterized builder

If you want to control what gets instanced based on a parameter:

public class ConcreteWidgetBuilder implements IWidgetBuilder {

  [Inject]
  public var injector:IInjector;

  public function buildPart(parameterised:IThing):IWidget {
    var interface:Class;

    switch(parameterised.style) {
      case ThingConst.COOL:
        interface = ICoolWidget;
        break;
      case ThingConst.HOT:
        interface = IHotWidget;
        break;
      default:
        interface = IDefaultWidget;
    }

    var widget:IWidget = injector.getInstance(interface);
    widget.param = parameterised;

    return widget;
  }

}

Bit freakin’ sexy if you ask me.

Aug 11

Visfleet’s Radiator (v 0.0.1)

I started a project not too long ago to implement an information radiator here at Visfleet. The first milestone has been reached: Show the status of important Hudson builds

And here it is:

The source is on github

I started out with the intention of using Flex 4 skinning to make a super-modern, slick looking radiator like Panic’s. But somewhere along the way I ended up with a two tone border, it reminded me of my Commodore days, and the rest is ugly :)

Near term plans include:

Then:

Aug 06

Best practice for Ruby’s require

Ruby’s require method gives you more than enough rope to hang yourself. Having now implemented too many projects with too many different approaches to using require, we thought it was worth writing down the conclusions we’ve come to.

We’ve gleaned most of our knowledge from these two articles, here, and here. So a big thank you to their authors.

How to require

Structuring your project

Dealing with executables

Guiding principle

If the above were boiled down to a single principle, it would be this:

Treat require like it’s requiring a namespace, and not a file. This is much the same as how import, and using, work in languages such as >C#, C++, and Java.

Jul 21

Another performance tool for AS3

Elad's performance monitoring tool for flash

Jul 17

Mediating ItemRenderers, Again.

I’ve posted about this before.

I still have no shame about mediating ItemRenderers and to my happy suprise, Flex 4 makes it simpler with the spark.ItemRenderer and its dataChange event.

Here’s my base prescription in sample form:

Now you and I both know it can be a lot simpler to achieve the same, namely:

Why mediate, it’s so simple? Well the reason is simple too. Things never stay simple. My prescription is a easy template for me to follow, its already second nature for me to do it that way.

The moment I need to mix selection state (or multiple models) into a single renderer, my mediator is there waiting. This includes ensuring that no matter what data source is generating the itemRenderer, I have a templated method for locating the correct model (hence the comment in my mediator above).

Jun 18

Postgres routine (and not so routine) maintenance

Postgres is our DB of choice. We run all our vTrack and vWork systems on it. We have lately been developing data archiving and culling strategies to manage data growth and disk consumption.

I’ve found a few hints that are best shared.

Culling Old Data to Recover Disk Space

The initial thought is to perform a quick delete of the old data, a-la:

DELETE FROM table1 where date < [some date in the past]

Nothing wrong with that. However, it doesn’t recover any disk space. The natural instinct is to follow it with a vacuum:

VACUUM VERBOSE ANALYZE table1

I like to throw verbose and analyze in there too. But doh, this doesn’t recover any disk space either, it just marks the space used by the deleted as free, making it available for new data later.

Which leads to what the Postgres manual says about the subject:

The FULL option is not recommended for routine use, but might be useful in special cases. An example is when you have deleted or updated most of the rows in a table and would like the table to physically shrink to occupy less disk space and allow faster table scans. VACUUM FULL will usually shrink the table more than a plain VACUUM would.

Ok, let’s do a VACUUM FULL then:

VACUUM FULL VERBOSE ANALYZE table1

Good; Yay; finally some improvement. But that took bloody ages on the massive table that I just removed a bazillion rows from.

So, we devised a cunning plan, loosely based on what another page in the Postgres manual says about recovering disk space. In the manual it says you should perform a TRUNCATE on the table if you want to clean it up quickly, which got me thinking. Instead maybe we should just copy all of the keep-able data into a new table and blow away the old one (along with all the cull-able data).

In it’s basic form:

CREATE TABLE table1new
SELECT * INTO table1new FROM table1 WHERE date >= [some date in the past]
DROP TABLE table1
ALTER TABLE table1new RENAME TO table1

Because SELECT INTO doesn’t bring across any of the table meta-data, we then put the correct restrictions on the columns manually:

ALTER TABLE table1 ALTER COLUMN column2 SET NOT NULL
ALTER TABLE table1 ALTER COLUMN column3 SET NOT NULL
....

… recreate primary keys and indexes:

ALTER TABLE table1 ADD CONSTRAINT table1_pkey PRIMARY KEY(column1)")
CREATE UNIQUE INDEX index_table1_on_column2_and_column3 ON table1 USING btree (column2, column3)
....

… find out the max integer for sequence re-creation. A bit of psuedo code here, depends on your scripting language (we wrap all this stuff around Ruby/Rails and ActiveRecord):

max_id = SELECT max(column1) FROM table1

Then re-create the sequence:

CREATE SEQUENCE table_column1_seq INCREMENT 1 MINVALUE 1 MAXVALUE 9223372036854775807 START #{max_id+1} CACHE 1 OWNED BY table1.column1")
ALTER TABLE table1 ALTER COLUMN column1 SET DEFAULT nextval('table1_column1_seq'::regclass)

All of this work just to save some disk space? Not to mention the fact that you are dropping a table, which has some risk to it - especially if everything breaks half way through through the process.

It sounds scary and seems like quite a lot of work, but the elapsed run time difference is pretty huge.

When doing a DELETE FROM followed by a VACUUM FULL, our test system took 130 seconds. In comparison, doing a CREATE TABLE, SELECT INTO, DROP TABLE and RENAME takes around 45 seconds. This is on a very small table (2M rows) by our standards.

Another difference between the two methods is that the first method keeps the table available for reads. Writes will be locked out because a VACUUM FULL takes out an exclusive lock on the table. The second method drops the table entirely, so an outage would be required for any services or daemons which are accessing that table.

If you think about it some more though, the fact that you are having to cull data out of the table means that it is probably a write-heavy table. This indicates that an outage would have been necessary anyway, to perform a VACUUM FULL. Given this, you might as well perform the faster (albeit more complicated) form of the cull.

A final note about VACUUM FULL and indexes

I also stumbled upon another little gem while seeing just how much disk we could save. If you do a VACUUM FULL it doesn’t also hoover the indexes.

Quote, from here:

The FULL option does not shrink indexes; a periodic REINDEX is still recommended. In fact, it is often faster to drop all indexes, VACUUM FULL, and recreate the indexes.

Pretty much says it all really. And it brings me back to the previous section about copying the data into a new table, and re-creating the indexes from scratch.

If you want to save a few MB without being scary however, a REINDEX might be a good command to keep in mind.