Aug/100
Presentation: Moose Best Practices
I wrote this for the TO.pm meeting on August 11th, 2010:
Jun/100
Presentation: Tools of the CPAN Ninja
I wrote this for the TO.pm meeting on June 10th, 2010:
Jun/100
BLOBs, Synonyms, and DBD::Oracle
INSERTing, UPDATEing, and SELECTing LOBs (CLOB/BLOB) in Oracle using Perl can be a PITA. Case in point: 2 years ago a fellow coworker ended up writing some extremely complex code using ora_lob_write(), etc, when all he really needed to do was use bind_param() and declare that the blob column is a blob so that DBD::Oracle would know to treat is specially. At the time I believe he did try something like:
INSERT INTO some_table (color, blob_data)
VALUES (?, ?)
]);
$sth->bind_param( 1, 'red' );
$sth->bind_param( 2, $data_for_blob, {ora_type => ORA_BLOB } );
$sth->execute();
But then an error like “DBD::Oracle::st execute failed: ORA-04043: object some_table does not exist (DBD SUCCESS: OCIDescribeAny(view)/LOB refetch)” was raised. Which made no sense since some_table is very much an object and any number of SQL operations work on it… when BLOBs are not involved. In the end he ended up having to write code that was triple the size to get around this issue.
So, 2 years later and a little hair pulling, I come to figure out by luck that the problem is that the underlying DBD::Oracle code, that makes dealing with BLOBs simple, fails when you are using a synonym in your insert. All we had to do is fully spell out the table name, and it works!
INSERT INTO some_schema.some_table (color, blob_data)
VALUES (?, ?)
]);
$sth->bind_param( 1, 'red' );
$sth->bind_param( 2, $data_for_blob, {ora_type => ORA_BLOB } );
$sth->execute();
Hopefully this helps others.
May/101
Generating Constants in Perl
The other day @ $work I was throwing together a module that creates constants based on a fairly static table in the database. Of course I didn’t want to hard code the contants, I wanted them to be magically created by what was in the database. For the sake of this example I had a table called “toy_categories” where each record has a unique ID (toy_category_id) and a name that can only be letters and underscores. The data would look something like:
-- --------------
12 dolls
7 action_figures
92 education
And the resulting constants would look something like this:
TOY_CATEGORY_ACTION_FIGURES = 7
TOY_CATEGORY_EDUCATION = 92
If you read the core perl documents you might be lead to think that the constant pragma is the way to go. This is not the case – just because a particular library is distributed with Perl does not mean its a good tool, instead it usually means that the library cannot be removed or substantially modified for backwards-compatibility reasons. In my experience most core libraries are to be avoided – there are much better solutions on CPAN.
So, the CPAN solution to constants is Readonly. This module hooks in to Perl’s ability to mark a variable as read-only, much like how variables can be flagged as tainted or scalars as UTF. Make sure you grab Readonly::XS as well to get the full benefit of read-only variables without the performance hit.
So, if you were to create a module for these constants you might do something like this:
use strict;
use warnings;
use Readonly;
use Exporter qw( import );
our @EXPORT = qw(
$TOY_CATEGORY_DOLLS
$TOY_CATEGORY_ACTION_FIGURES
$TOY_CATEGORY_EDUCATION
);
Readonly our $TOY_CATEGORY_DOLLS => 12;
Readonly our $TOY_CATEGORY_ACTION_FIGURES => 7;
Readonly our $TOY_CATEGORY_EDUCATION => 92;
1;
And then in some module you can access these constants:
if ($toy->category_id() == $TOY_CATEGORY_EDUCATION) {
my $response = ask_question('Are you an educator?');
}
Now, like I said, I don’t want to hardcode the constants. I want them to be dynamically created by records in my toy_categories table which resides in my database. Its actually pretty simple to do this with some tricky evals:
use strict;
use warnings;
use Readonly;
use Exporter;
use Exporter qw( import );
our @EXPORT;
{
my $dbh = code_that_returns_a_dbi_handle();
my $sth = $dbh->prepare(q[
SELECT toy_category_id, name
FROM toy_categories
]);
$sth->execute();
$sth->bind_columns( \my( $id, $name ) );
while ($sth->fetch()) {
_export_variable( "toy_category_$name" => $id );
}
}
sub _export_constant {
my ($variable, $value) = @_;
$variable = '$' . uc($variable);
my ($error, $failed);
{
local $@;
$failed = not eval("Readonly our $variable => \$value");
$error = $@;
}
if ($failed) { die "Unable to create constant $variable: $error"; }
}
1;
What’s going on here? Let’s start at the top:
This is the least intrusive way of using exporter and doesn’t pollute your namespace nearly as much as ‘use base qw( Exporter );’ does and is friendlies to other modules.
Replace this with whatever you use to get a DBI database handle. Take a look at DBIx::Connector for a great alternative to doing this directly with DBI.
SELECT toy_category_id, name
FROM toy_categories
]);
$sth->execute();
$sth->bind_columns( \my( $id, $name ) );
while ($sth->fetch()) {
...
}
I’m a big fan of writing my DBI selects in this fashion with bind_columns() because it tends to be the fastest way to get the data (versus fetchrow_hashref, etc) and tends to lead to the simplest code within the while loop since it just needs to use $id and $name versus $row->{$id} and $row->{name} (for example).
While all of the code within _export_variable() could just be inlined right in the spot, that’s bad design. If you can get a piece of your code generalized and moved to a subroutine, do it.
Constants should always, due to convention, be uppercase. It is very important that you code to popular conventions because other people will most likely be working on your code later and if you come from a common expectation of how various things are done they will have an easier time ramping up to your code. I try to code to the Modern Perl / CPAN conventions.
This bit of code was taken from nothingmuch’s blog entry where he announces Try::Tiny which provides a safe way to handle eval errors. Read up there if you want to understand why the code was written this way.
While this code is several blocks deep in scope, this constant will exist in the package’s scope since it is being declared with ‘our’.
That’s it. After this was implemented I thought it was so useful, simple, but requiring the knowledge of a few tricks, that I’d share it with ya’all. Enjoy!
Apr/101
YAPC::NA 2010 Talks are Being Approved
I just got my approval for the two proposals I had submitted for this coming YAPC::NA 2010! I’m very excited. This will be my first time doing any sort of talk at a YAPC. I’ve been wanting to do this for years and I am very grateful that the organizers of the conference considered me. Over the last few years I’ve been trying to hone my presentation skills by presenting various topics almost monthly at the Thousand Oaks Perl Mongers and Los Angeles Perl Mongers.
The first talk is a five-minute lighting talk about how to clone yourself in Perl. For those that are unaware: lighting talks are a great YAPC tradition where around a dozen speakers will present 5-minute or less talks. These talks range from very advanced computer science topics, down to the silly, and sometimes bizarre.
The other talk I’m doing is a 40-minute lecture/discussion showing examples of writing concise code. This will be an extended version of the similar presentation that I’ve given at mongers. I’ll be delving in to real-world cases where writing code in a more concise manner, reduces technical dept, leads to more robust code, and a whole lot more.
See you at YAPC!
Apr/100
Cassandra Presentation
I wrote this for the TO.pm meeting on April 14th, 2010:
Apr/106
Apache Cassandra and the Thrift API on CPAN
I’m currently playing around with a project that uses Apache Cassandra. The Cassandra website states:
“The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo’s fully distributed design and Bigtable’s ColumnFamily-based data model.”
So, from what I’ve experienced so far this is either a distributed Berkely DB, or a persistent Memcached, depending on which concept you are more familiar with (I know, that’s incredibly over simplified).
Cassandra was developed by Facebook and was made open source several years ago. It uses the Thrift API to communicate, which was also developed by Facebook. Supposedly there are some steps being taken to move Cassandra on to a different API, but Thrift is the defacto for now.
There are two modules on CPAN that deal with Cassandra. Net::Cassandra is the original, and in my opinion, the superior/simpler/easier module. Net::Cassandra::Easy is a later implementation that really isn’t any simpler from what I can tell and neither seems any more or less tied to the intricacies of the Thrift API despite Net::Cassandra::Easy’s accusations of Net::Cassandra. Both implement the Thrift API under the scenes in their own way, and both are lacking in features.
Now, the Thrift API itself is distributed with a perl library called simply “Thrift”. This library is very new and it shows. It needs a lot of work, a lot of automated tests, and IMO should be on CPAN. I consider it a fail when a project decides to distribute Perl libraries independently from CPAN. This leads to very little exposure – I was lucky I found the perl library in the Thrift tarball, I doubt others have been as lucky.
So, I’d like to make a Thrift driver library and put it on CPAN. Then, a truly Thrift independent Cassandra client library can be written that uses the Thrift library behind the scenes, but is not directly tied in with it. Later, when Cassandra decides to change their API library, the Cassandra library can be changed to use a new driver.
An additional benefit to this is if any Cassandra features are not yet available via the Cassandra module, then the perl developer can easily drop down to the Thrift driver and directly write Thrift queries.
Has anyone thought of this, or perhaps begun working on something like this? If not, I’d like to do it and I’d love to hear any suggestions and thoughts people have.
Jan/105
Robust DBI Transaction and Connection Handling
Edit: It turns out there is already a module on CPAN that does exactly what I talked about here. Its called DBIx::Connector. I haven’t tried it yet, but it looks like the guy that wrote it designed it with input from the DBIC guys.
I’m a big fan of DBIx::Class. Among DBIC’s many great features is its superb transaction and connection handling via DBIx::Class::Storage::DBI. When I’m using raw DBI I feel like I’m missing these core components. I should always be able to expect robust transaction and connection handling.
I started playing around with creating a new CPAN module (something like DBIx::Robust, DBIx::Transaction, etc…) but each time I dove in to it I kept coming to the conclusion that DBIC’s DBI storage drivers provide everything I need and I would just be shooting myself in the foot by either porting DBIx::Class::Storage::DBI to a DBIC independent API or by writing it from scratch.
Before I go any further, let me show you how attractive and awesome DBIC’s storage layer is:
- Transactions may be nested, even without savepoints.
- Transaction savepoints can automatically be tracked allowing for reliable and incremental rollbacks.
- The appropriate DateTime::Format class is automatically determined based on the type of database.
- All database calls efficiently pass through a layer that detects stale connections that will attempt to re-connect to a database.
“There is no charge for awesomeness, or attractiveness.”
The DBIx::Class::Storage::DBI pod illustrates all this in a round-about way: “If you set AutoCommit => 0 in your connect info, then you are always in an assumed transaction between commits, and you’re telling us you’d like to manage that manually. A lot of the magic protections offered by this module will go away. We can’t protect you from exceptions due to database disconnects because we don’t know anything about how to restart your transactions. You’re on your own for handling all sorts of exceptional cases if you choose the AutoCommit => 0 path, just as you would be with raw DBI.”
This stuff is really powerful, and we should never have to work with databases in Perl without it. But, currently, it is only meant to work under DBIC, which is a shame. I’m betting that DBIx::Class::Storage::DBI could be used directly, bypassing DBIC completely. There is some DBIC stuff layered in there, but that should be ignorable.
What do the rest of you think? Has anyone considered moving this logic out in to a generic distribution that isn’t DBIC specific? Is there merit in what I’m talking about? Does anyone use DBIx::Class::Storage::DBI directly, bypassing DBIC?
Oct/090
use HTML::FormHandler;
I’ve been meaning to give HTML::FormHandler a try for quite some time now. Over the weekend I converted a form from some home-grown form validation and html generation to use HTML::FormHandler instead. HTML::FormHandler was inspired by, and quasi-forked from, Form::Processor. There are many options for processing forms, such as FormValidator::Simple (Catalyst’s default) and Data::FormValidator, to name a few.
Up until now I’ve always thought that the existing form handling/validating modules on CPAN were incredibly lacking. They all have these crazy and inconsistent APIs that in the end limit what you are able to do. I don’t expect a form validation API to handle every possible situation, but I do expect it to provide a consistent API that is well documented and allows me to easily extend if I need additional functionality. Plus none of them use Moose (AFAIK), which isn’t just me being a Moose evangelist, its also practical because Moose provides mechanisms to extend functionality via roles and to provide reusable validation with MooseX::Types.
HTML::FormHandler solves these issues. HTML::FormHandler can be a complete end-to-end solution for declaring form fields and types, applying server-side validation, generating the HTML for a form, and applying form submits to the database. Almost everything and the kitchen sink is there, except client-side form validation, which can be added separately of course. Normally a “do everything” package scares me, especially for something like form validation. But, Gerda Shank (the author of HTML::FormHandler) did a great job splitting out the various pieces of HTML::FormHandler so that any piece of the form handling process can be subverted to your own needs, and there is copious amounts of documentation explaining how to do so.
The HTML::FormHandler::Manual::Intro does a great job introducing HTML::FormHandler.
There are a few downsides to this module’s current state. It is relatively new and is still stabilizing. For example, just recently the results of form validation were separated from the form object itself. This is a huge fundamental change, but a necessary one. Also, the concept of widgets was recently introduced. Widgets are a great way to extend and customize the display of form fields, and the form as a whole. But, this was just introduced and is still experimental, and has some rough edges. Despite this, the widget design is very promising and I ended up using it myself (thanks to HTML::FormHandler::Manual::Rendering) as my preferred rendering method.
Enjoy.
Sep/093
Top 10 Reasons to use Perl
You should use Perl because…
10. Real Perl coders don’t have fohawks.
9. Perl has nothing to do with Ryan Seacrest.
8. Beer improves Perl.
7. It comes with a 20-sided die for all that TIMTOWTDI.
6. It has nothing to do with PHP.
5. Captain Picard writes perl.
4. There is no better place to get Moo seX.
3. It grows on you.
2. IE8 was not written in Perl.
… drum roll …
1. You can write this.
