Agile Perl (and other technologies)

Deep thought

2010-06-30T14:37:00.001-07:00

Quick and dirty code is seldom quick in the long run. It is, however, always dirty.

Perlmonks is a bit of an embarrassment

2009-07-30T09:09:00.000-07:00

The site is painfully slow, not really a good advertisement for Perl as a web implementation language. And now the latest revelation is that passwords were stolen because they were stored in plaintext. I hate to say it, but perlmonks is probably a significant reason why Perl is a language in danger of dying out. Not just the technical issues (although perlmonks is a painfully difficult site to use), but cultural as well as there's a rather hierophantic attitude on the site, maybe no worse than at similar sites of other technologies, but my impression is that it is.

Stripes

2009-07-07T16:25:00.000-07:00

I came across a reference to Stripes while trying to figure out how to handle some mapping issues in Spring MVC, so I'm taking a look at it and trying to use it in place of Spring MVC on the view part of a learning app I'm building. My property mapping question is explicitly answered in the documentation, which makes me feel comfortable with the prospect of working with Stripes, although the instructions for installing and using the Stripes archetype failed badly in my environment. I'll come back to this at some point in the future, I suppose, assuming that Stripes does all that I would like it to and I'll skip that. But in the meantime, it doesn't appear that I really need it for what I'm doing, so I'll happily continue on my way as I develop my app.

CSS z-index not working

2009-06-04T16:24:00.000-07:00

It's worth noting that z-index is only applied to absolutely positioned elements.

Arg, Perl attributes don't do what I want them to

2009-03-13T21:34:00.000-07:00

Apparently, there is no way to get the name of a non-subroutine with an attribute. That means that if I want to annotate fields in a class for serialization, I need to identify the column for each one. On the plus side, I guess that means that I can move pass this particular hurdle (for now, at least).

Time to start again

2009-03-12T08:51:00.000-07:00

I had thought that Attribute::Handlers was going to be the quick and easy solution to dealing with attributes for my ORM, but it turns out that it can only get the name of a subroutine with an attribute, not arrays (or hashes). I'm thinking that perhaps the solution is to (old-school) subclass Object::InsideOut and integrate with its attribute parsing code... This would also enable me to see the :Type attribute from OIO.

A roadmap for my Perl ORM

2009-02-16T18:10:00.000-08:00

Mostly for my own purposes (and subject to modification).

0.1 Handle serializing a simple OIO class
0.2 Handle de-serializing a simple OIO class
0.3 Allow embedding classes (1-1 relationship) in a class (serialize/deserialize)
1.0 Allow 1-many, many-1, many-many relationships between classes (to separate tables, using join table if necessary)

Thoughts on ORM in Perl

2009-02-10T21:50:00.001-08:00

One thing that I really admire from Java world is the EJB3 Persistence annotations. A class can be made persistent with a simple mark-up:


@Entity
class Foo {
   private int count;
   private String name;
   @Transient
   private int temp_var;
   ....
}

The @Entity notation indicates that the class should map to a table (unless overridden, it will use the class name as the table name). Every field then automatically maps to a column in the table (column names can be overridden) unless marked with the @Transient annotation.

I was thinking that something similar could be executed in Perl using perl attributes. I could do something like


package Foo;
use Object::InsideOut;

my @int :Field :Entity;
my @string :Field :Entity;
my @temp_var :Field;
...

Using something like Object::InsideOut seems essential for this sort of attribute-based mark-up, although perhaps I should look at Moose instead (although that does seem to have some significant performance penalties). There are some questions about how to handle exporting the serialization routines. Should I work towards a DAO solution or set up serialization routines directly in the class (perhaps as protected by default with the class making a decision about whether to provide public methods to expose these to clients.

A tutorial on attributes

2009-02-03T17:22:00.000-08:00

The perldoc on attributes is a bit opaque, so digging about for information on how to program attributes in Perl, I managed to come across slides from a talk at london.pm Using Attributes Effectively (PDF). It suffers a bit in being only the slides and not the talk, but it does cover some things (like the handler functions) that seem to be largely undocumented elsewhere.

MySQL bench marks, comparing MyISAM to InnoDB and evaluating the performance hit of foreign keys

2009-01-29T22:36:00.000-08:00

During an interview earlier this week, the question of how using foreign keys would impact performance on MySQL. I decided that it would be worthwhile to actually check this out in practice. I started out with a basic structure as follows:


CREATE TABLE IF NOT EXISTS `a` (
  `id` int(11) NOT NULL,
  `data` varchar(255) NOT NULL,
  `data2` varchar(255) NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `b` (
  `id` int(11) NOT NULL,
  `a_id` int(11) NOT NULL,
  `data3` varchar(255) NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Then added a second database where we had a foreign key:


CREATE TABLE IF NOT EXISTS `b` (
  `id` int(11) NOT NULL,
  `a_id` int(11) NOT NULL,
  `data3` varchar(255) NOT NULL,
  PRIMARY KEY  (`id`),
  KEY `a_id` (`a_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

ALTER TABLE `b`
  ADD CONSTRAINT `b_ibfk_1` FOREIGN KEY (`a_id`) REFERENCES `a` (`id`) ON DELETE CASCADE;

And just to make things complete, I added a third database identical to the first, but using MyISAM as the engine.

The first thing is to do a data population. I wrote a perl script to generate 10,000 inserts of random data into table A, each with 0-5 random inserts into table B.
Next a selection of queries. I did 500 variations of SELECT * FROM a INNER JOIN b ON (a.id=b.a_id) WHERE a.id=?

For deletes, I deleted 500 rows, using DELETE FROM a WHERE id=?; DELETE FROM b WHERE a_id=? for the databases without the foreign key and the simple DELETE FROM a for the foreign key database.

The results for inserts:

database	insert time (s)	select time (s)	delete time (s)
InnoDB, no FK	4.5	0.25	3
InnoDB, FK	4.0	0.0	1
MyISAM	1.0	0.75	0

I had always assumed that the advantage of MyISAM over InnoDB was the opposite of what it turned out to be with this benchmark. Perhaps even more surprisingly, the foreign key version of the InnoDB database was uniformly faster than the non FK version. I would have expected to see some overhead at least in the insertion stage, but even on multiple trials, I found a consistent advantage for the foreign keys on the insertion.

Digging back in

2009-01-17T20:23:00.001-08:00

Having recently been downsized from oversee.net, I'm using some of the time to revisit some of my old projects. At the top of the stack is lalocalfood.com, where the first phase is an improved version of my map of Los Angeles farmer's markets. Coming back to some of the old code, it's interesting to see how much time is necessary to recover my memories of some of this code I haven't touched in years. That said, once I get over that, I'm finding that my code has surprisingly good design where very little effort is required to handle new cases. I should be finished with the scraper code to collect the farmer's market data from the two main sites with one more day's work.

Creating a controller in Catalyst

2008-04-24T21:33:00.000-07:00

A new controller in Catalyst is registered by running script/APPNAME_create.pl CONTROLLER_NAME. This creates a module in the controller directory and a test in the test directory. These are apparently all that is needed for a controller to be created.

It's worth noting that /foo can activate either App::Controller::Foo::index() or App::Controller::Root::foo(). However, if both exist, App::Controller::Foo::index() takes precedence.

Module Monday: Quantum::Superpositions

2008-04-21T21:09:00.000-07:00

A new weekly feature. I'm going to dig out some of the hidden gems of CPAN. One of the best places to look is Damian Conway's list of modules and this is one fun and interesting one.

Here's a selection of what Quantum::Superpositions can do:


$wanted = any("Mr","Ms").any(@names);
if ($name eq $wanted) { print "Reward!"; } 

$okay = all(\&check1,\&check2);
die unless $okay->();

my $large =
         all(    BigNum->new($centillion),
                   BigNum->new($googol),
                   BigNum->new($SkewesNum)
          );
@huge =  grep {$_ > $large} @nums;

Installing Rose::DB::Object

2008-04-20T01:20:00.000-07:00

For the Backlog project, I was thinking this was going to be a good opportunity to also learn Rose::DB::Object, so I tried installing it via CPAN. Oops, it had some errors along the way. It turns out that SQL::ReservedWords has a dependency on Data::OptList which it doesn't declare. Installing Data::OptList solved that problem, but Rose::DB::Object is still failing its test suite:


t/db-object-manager..................7/3900 Can't locate object method "format_select_start_sql" via package "Rose::DB::__RoseDBPrivate__::Rose::DB::MySQL" at /Users/dhosek/.cpan/build/Rose-DB-Object-0.769-AXgKMz/blib/lib/Rose/DB/Object/QueryBuilder.pm line 725.
# Looks like you planned 3900 tests but only ran 827.
# Looks like your test died just after 827.
t/db-object-manager.................. Dubious, test returned 255 (wstat 65280, 0xff00)
 Failed 3073/3900 subtests 
 (less 783 skipped subtests: 44 okay)

My insomnia feels resolved, so I'll come back to this after sleep.

Update (21 April): I managed to solve the problem: Doing an install of Rose::DB::MySQL then re-trying to install Rose::DB::Object managed to do the trick.

Learning Catalyst: The Backlog project. Day 1

2008-04-18T21:20:00.000-07:00

The Plan

What I'm looking to build is a web application which is designed for maintaining a product backlog. I see the following as the key user stories:

As a programmer, I want the system to use the wiki as its primary display and authentication system (so, users will log into Backlog using their wiki credentials, and the backlog is visible from the wiki).
As a programmer, I want the system to import the existing backlog from the wiki.
As a User, I want to be able to submit stories to the product backlog.
As a product owner, I want to be able to prioritize and arrange the backlog.

Installing Catalyst

This went far more easily than I expected. A simple CPAN install brought it in with no problems. I guess I was just lucky on that front.

Building the core

Step one was to build the core of the application. I ran catalyst.pl from the command line and got the following:


~/dev/Backlog> catalyst.pl Backlog
To use the Catalyst development tools including catalyst.pl and the
generated script/myapp_create.pl you need Catalyst::Helper, which is
part of the Catalyst-Devel distribution. Please install this via a
vendor package or by running one of -

  perl -MCPAN -e 'install Catalyst::Devel'
  perl -MCPANPLUS -e 'install Catalyst::Devel'

Apparently the Catalyst package is divided into two parts, one just for running Catalyst, one for developing Catalyst applications. Fortunately this install was also uneventful.

Rerunning it, I got


~/dev/Backlog> catalyst.pl Backlog
created "Backlog"
created "Backlog/script"
created "Backlog/lib"
created "Backlog/root"
created "Backlog/root/static"
created "Backlog/root/static/images"
created "Backlog/t"
created "Backlog/lib/Backlog"
created "Backlog/lib/Backlog/Model"
created "Backlog/lib/Backlog/View"
created "Backlog/lib/Backlog/Controller"
created "Backlog/backlog.yml"
created "Backlog/lib/Backlog.pm"
created "Backlog/lib/Backlog/Controller/Root.pm"
created "Backlog/README"
created "Backlog/Changes"
created "Backlog/t/01app.t"
created "Backlog/t/02pod.t"
created "Backlog/t/03podcoverage.t"
created "Backlog/root/static/images/catalyst_logo.png"
created "Backlog/root/static/images/btn_120x50_built.png"
created "Backlog/root/static/images/btn_120x50_built_shadow.png"
created "Backlog/root/static/images/btn_120x50_powered.png"
created "Backlog/root/static/images/btn_120x50_powered_shadow.png"
created "Backlog/root/static/images/btn_88x31_built.png"
created "Backlog/root/static/images/btn_88x31_built_shadow.png"
created "Backlog/root/static/images/btn_88x31_powered.png"
created "Backlog/root/static/images/btn_88x31_powered_shadow.png"
created "Backlog/root/favicon.ico"
created "Backlog/Makefile.PL"
created "Backlog/script/backlog_cgi.pl"
created "Backlog/script/backlog_fastcgi.pl"
created "Backlog/script/backlog_server.pl"
created "Backlog/script/backlog_test.pl"
created "Backlog/script/backlog_create.pl"

running ./Backlog/script/backlog_server.pl started up the web server on port 3000 and I was able to verify that I had a working Catalyst application to begin playing with.

The @INC array can be more dynamic than you think

2008-04-17T20:23:00.000-07:00

We can put subroutine references, array references and objects into the @INC array. See perldoc -f require for the hairy details.

Test-first doesn't mean every test must be written before you start coding

2008-04-15T13:42:00.000-07:00

It's worth noting that test-first doesn't mean that all the tests have to be written before any code is written. In fact, in some cases, it's actually impossible to write the test before the implementation is at least partly scoped out (e.g., when it's necessary to mock objects or modules external to what you're testing). A lot of test cases end up coming up during the development process. Just like elsewhere in agile development, the secret is in tight loops. Write the test as soon as possible, but no sooner than necessary.

Installing DBD::mysql on OS X Leopard 10.5.2

2008-04-07T19:11:00.001-07:00

One of those minor nightmare installs. I need this though, so I've gotta work for a while.

Here is the problem that shows up:


~/.cpan/build/DBD-mysql-4.006-j5Hpiv> prove -v -l t/00base.t
t/00base......
1..5
ok 1
ok 2
ok 3
install_driver(mysql) failed: Can't locate loadable object for module DBD::mysql in @INC (@INC contains: /Users/dhosek/.cpan/build/DBD-mysql-4.006-j5Hpiv/lib /Users/dhosek/.cpan/build/DBD-mysql-4.006-j5Hpiv/lib /System/Library/Perl/5.8.8/darwin-thread-multi-2level /System/Library/Perl/5.8.8 /Library/Perl/5.8.8/darwin-thread-multi-2level /Library/Perl/5.8.8 /Library/Perl /Network/Library/Perl/5.8.8/darwin-thread-multi-2level /Network/Library/Perl/5.8.8 /Network/Library/Perl /System/Library/Perl/Extras/5.8.8/darwin-thread-multi-2level /System/Library/Perl/Extras/5.8.8 /Library/Perl/5.8.6 /Library/Perl/5.8.1 .) at (eval 3) line 3
Compilation failed in require at (eval 3) line 3.
Perhaps a module that DBD::mysql requires hasn't been fully installed
 at t/00base.t line 38
 Dubious, test returned 2 (wstat 512, 0x200)
 Failed 2/5 subtests 

Test Summary Report
-------------------
t/00base.t (Wstat: 512 Tests: 3 Failed: 0)
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 5 tests but ran 3.

~~First step, is that I'm finding that the blib/arch/auto/DBD/mysql/mysql.so file is not being created.~~

Actually, that's not the problem! Looking closer, I'm seeing that the .bs and .bundle files are apparently the ones that apply now. Interestingly using prove -b t/00base.t does in fact work. But make test has the same test fail...

Ignoring that issue, I move on to the next test:


prove -b t/10connect.t 
t/10connect......dyld: lazy symbol binding failed: Symbol not found: _mysql_server_init
  Referenced from: /Users/dhosek/.cpan/build/DBD-mysql-4.006-j5Hpiv/blib/arch/auto/DBD/mysql/mysql.bundle
  Expected in: dynamic lookup

dyld: Symbol not found: _mysql_server_init
  Referenced from: /Users/dhosek/.cpan/build/DBD-mysql-4.006-j5Hpiv/blib/arch/auto/DBD/mysql/mysql.bundle
  Expected in: dynamic lookup

t/10connect...... Failed 2/2 subtests 

Test Summary Report
-------------------
t/10connect.t (Wstat: 5 Tests: 0 Failed: 0)
  Parse errors: Bad plan.  You planned 2 tests but ran 0.
Files=1, Tests=0,  0 wallclock secs ( 0.01 usr  0.00 sys +  0.04 cusr  0.00 csys =  0.05 CPU)
Result: FAIL

Update: 11 April 2008 The problem has its roots in using the pre-compiled distribution of mysql from mysql.com rather than building from source. This resulted in getting incompatible libraries between what was compiled for DBD::mysql and what I had in the mysql directory. Building from source resolved the problem. Good instructions on doing the job are available at HiveLogic.

Getting Things Done

2008-03-07T18:59:00.000-08:00

I first became aware of Getting Things Done from the 43 folders web site. While I've had the book for a couple years, I finally read it this week (along with installing OmniFocus on my work Mac). So far, I'm finding the GTD workflow to be quite helpful. Omnifocus, somewhat less so, other than as a trusted repository for projects. The big difficulty is that Omnifocus ties me too much to the Mac. On the other hand, there has been an announcement that OmniGroup will be doing an iPhone version of Omnifocus. That plus multi-computer syncing could make Omnifocus a really great product for my needs.

First experiences with OS X Leopard (10.5)

2008-03-05T17:36:00.000-08:00

I've finally got the iMac with Leopard on my desk at work. So far it's pretty good, although I do have a few complaints, most notably spaces got a few things wrong: cmd-tab and cmd-` should never change spaces. It's really disconcerting to have that happen. I want to be able to pretend that what's in the current workspace is everything I've got, so I can do quick and easy context switches.

And the choices for the switch between spaces are really disappointing. I don't think that I have an app which doesn't use one of ctrl-arrow, opt-arrow or cmd-arrow for something. At this point, I've just disabled the arrow switching.

Version control and databases

2008-03-04T17:21:00.001-08:00

We have a practice of having a branch for each story in the sprint. This works fine with most stories, but what happens when we have stories that affect the database. I found this discussion on the topic which has some interesting reflections and comments. In general, though, my sense is that version control for databases is at best ad hoc and kludgy. RIght now we're doing something similar to Rails migrations, but handled manually. I'd like to formalize that process. I'm looking at DBIx::Migration::Directories, although I find no commentary on it at all, anywhere.