Micky Dionisio's Software Architecture Blog

Talk Techy to Me - Your favorite tech feeds read to you!

2011-03-16T08:05:00.000-07:00

Hey my fellow tech geeks. Just like you, I love tech. I live and breathe it. It consumes my 9-5 and 5-9. But if your life and job is as crazy as mine, you know that it's tough finding time to catch up on all your tech feeds. If we fall behind for even a couple of days you start to feel guilty and "disconnected". So I sought out a way to try and change that.

My commute to the Yahoo! campus in Burbank, CA takes about 45 minutes (that's on a good day!). I often load up on audiobooks but sometimes I have A.D.D. and can't focus on listening to one topic at length. Music and radio, get old pretty quick too. While I do appreciate the way that Ryan Seacrest can talk about nothing and make it sound interesting I can't handle it for more than a 15 minutes.

One day as I was driving home from work it hit me that my time on the road could be better spent catching up my tech feeds. So in classic programmer spirit, I set out to create a mobile app that would read my favorite tech blogs to me while I'm on the road.

So came the birth of Talk Techy to Me. This version is free of charge while I work on the paid version that is fully customizable. At the time of this posting it currently supports six popular tech feeds:

1) TechCrunch
2) Official Google Blog
3) High Scalability (for my programming peeps!)
4) Mashable
5) ReadWriteWeb
6) VentureBeat.

Give it a whirl, spread the word, and send me some feedback. Remember it's in beta so when you find bugs please contact me and I'll get it sorted out ASAP. And hey, if you like the app and find it useful, a good review wouldn't hurt ;)

I guess this constitutes my foray into mobile development...

UPDATE - By popular demand, Engadget has been added to the list of free feeds!

-Mick

Talk Techy to Me
Talk Techy to Me in the Android market

Distributed Computing, Part 1

2011-01-10T09:43:00.000-08:00

Abstract

You've read or heard about them. Distributed computing frameworks or technologies like Hadoop or Google File System (GFS) and Google MapReduce. This series of posts aims to shed some light on when, why and how you would implement distributed computing techniques. If most of the systems you've encountered don't deal with storing and processing large amounts of data (terabytes and up) on a daily basis, or your company chooses to purchase 3rd party data storage and analysis services, then you may not have realized the critical advances that have been made this past decade in the distributed computing field.

Since this is part 1 of the series, I'll touch on the three most important problems that distributed systems aim to solve. In subsequent posts I’ll dive deep into what sparked it all - Google File System and Google MapReduce. Then we’ll explore an open source distributed computing platform based off of GFS and MapReduce. This platform called Hadoop, was created by Doug Cutting and spearheaded by Yahoo!. Hadoop is used by many technology giants such as Facebook, Twitter, Amazon and IBM to name a few. We’ll eventually take Hadoop out of the lot for a test drive.

Let’s get our feet wet.

Problem 1 - High-end servers are expensive

No company likes shelling out thousands of dollars for high end servers. 10 years ago, some could say that servers like these were a necessary evil since data collection is the life blood of any organization and all data must be collected at all costs. Enterprises today continue to intake an enormous amount structured and unstructured data per day and it's growing exponentially.

How do you begin to store massive amounts of data? If we take a look back to the early 2000’s, companies were spending some serious coin to either build massive fault tolerant storage systems from scratch or enlisting companies like IBM, HP, SAS etc. to help manage and analyze their data. Either way, you needed high-end servers that come with all the bells and whistles. These servers come with a very high total cost of ownership. It costs a fortune to acquire, a fortune to maintain and like always hardware will fail often and need expensive replacement.

On top of that, as data capacity limits red line more hardware needs to be acquired and added to the system. Sure, these systems are built for scalability but it's not easy explaining to your CIO that you need another rack full of expensive blades and switches. This type of horizontal scaling gets expensive very quick. Once you're invested in this type of technology stack you are pretty much locked in for the long haul and it's a vicious cycle. You can bank on your ROI margins dropping over time.

Problem 2 - It's tough to efficiently store massive amounts of data

I’m not talking about a few hundred thousand gigs. I'm talking about storing and processing terabytes, petabytes or greater of information on a daily basis. The amount of data that is passed over the internet alone is mind boggling. If you can imagine the birth of the internet as being the Big Bang then we haven’t even entered the cooling phase. That is how explosive and ever expanding the internet and modern systems continue to be. According to studies performed by to Cisco, as of March 2010 the amount of data that is exchanged across the internet per month is *drumroll*... 21 exabytes.

Wait, what? 21 exabytes per month?! If you were to store all of the words ever spoken by human beings since the dawn of time, it would take only 5 exabytes. Cisco supects that in 2013, that number of data exchanged per month on the internet will grow to 56 exabytes.

That’s a lot of data.

Problem 3 - It's tough to analyze and process massive amounts of data

Data comes in many formats such as text, audio, video and images that ranges from structured, semi-unstructured and unstructured. How would you begin perform analysis on this data to find patterns across different formats and types? For instance, how do you determine the patterns of a given user given his purchase history (structured data) and surfing behaviors (semi-structured log files)? It’s pretty tough... and costly... and computation could take a long time if resources aren't allocated in an efficient manner. Regardless, companies know that the answers to these questions provide insight that is paramount to their business. It can discover patterns that can be key to understanding markets, drive new strategies, improve business process, perform scientific calculations to enhance offerings etc.

That’s why companies that regularly deal with massive amounts of data, like Google, started looking for cheaper, more efficient solutions. Their systems needed to be scalable and elastic as ever.

Enter Google File System (GFS)

In 2003, Google published a paper detailing some of the science and design behind “The Google File System”. I’ll dive into the GFS in the next post but at a high level, this file system was developed to provide fault tolerant, reliable data storage using clusters of inexpensive commodity servers. Not the high-end servers that cost thousands of dollars. Just simple, affordable rack mountable Linux servers.

This design that Google developed directly solved problems #1 and #2.

Then Google MapReduce

The very next year after Google released it’s paper on GFS, they released another paper called “MapReduce: Simplified Data Processing on Large Clusters”. The purpose of their now patented MapReduce framework is to be able to process and generate large data sets across a cluster of commodity machines in a distrubuted fashion. In essence, it is always better to move computation to data rather than moving data to computation. This exploits the benefits of utilizing data locality in a clustered environment. It is modeled after the map/reduce strategies of functional programming wherein there are two phases, map then reduce.

During the map phase, computation is broken up into smaller chunks and sent to all data nodes in the cluster. Since data is distributed across the cluster, computation will be executed in a parallel fashion since each data node has its own subset of data. When each node is done with its computation, the resulting data set then sent to the master node.

The master node then starts the reduce phase. All information sent back from the data nodes are combined to get the original “answer” or output. In typical cases, this data is saved for processing by another program.

Google MapReduce solved problem #3 by providing faster computation and analysis using distributed techniques in tandem with GFS.

Conclusion

- There are three main reasons why’d you want to look into a distributed computing framework.

You need cheap, reliable, fault tolerant data storage that scales horizontally.
The amount of data you collect is growing exponentially. (terabytes, petabytes, exabytes and beyond)
You need to efficiently process and analyze high volumes of unstructured, semi-structured, and structured data.

- Although various distributed computing techniques have been around for decades they are often proprietary and expensive.

- Google's white papers on GFS and MapReduce sparked an open source distributed computing revolution (like Hadoop).

In the next posts, we’ll explore GFS and MapReduce then introduce you to Hadoop.

I hope this helps shed a little light on distributed computing and hope that you'll to read the next articles in the series. Until then, happy coding!

Micky

Finding Intersections Destructively / Nondestructively - Java Collections

2010-04-23T11:22:00.000-07:00

What is an intersection?

Mathematical set theory states an intersection is the common elements between given sets. It's stated as such, A ∩ B, where A and B are different sets.

Visually it looks like this:

When would you need to calculate intersects?

Given n collections, you need to identify only the elements they have in common and ignore the rest.

Admittedly, I'm a bit spoiled coming from a web technology wherein most cases you can just write up some snazzy SQL to find the commonality and return the record set. In a good sized Java application though you'll often have a good amount of objects and data stored using collections in cache, so SQL is out of the question. You may be tempted to go down the road and create some sort of method that looks like so numberOfOccurrences(Set, String)that returns the number of occurrences and you use that number to determine what to drop and what to keep. This could work but there is a better, more efficient way to achieve this using a built in Java method - retainAll() that resides in the Collections interface. Lets see how:

The code

We create two sets each holding fruits. You'll see that Apple and Orange are the common elements between the two sets. Make believe these two sets are your collections sitting in cache.
We create an HashSet called intersection to hold our common values. Since we are created a holding set, we are finding the intersection nondestructively, which means without modifying the any of sets being compared. Simply instantiate the intersection set passing in set1 as the initializer.
To find the intersection, all we do is call retainAll(Collection c) passing in the collection to be compared.
Then we print out the intersection results.
Next I show you how to destructively find the intersection by using set2 against set1, again using retainAll() and printing out the both sets. You'll see that set2 has been altered.

You should see the following results...

So, there you have it. Pretty painless right?

Helpful IntelliJ IDEA Shortcuts

2010-04-08T15:17:00.000-07:00

IntelliJ IDEA Shortcuts

After settling in at the new gig, the IDE of choice for my team is IntelliJ IDEA. I've been using Eclipse for many many years now so there has been growing pains because of the simple fact that I was extremely familiar with all the ins-and-outs of Eclipse and the shortcuts they offered. After a month or so of using IntelliJ, I'm getting more comfortable with it. I found myself writing these notes on a scratch pad and figured I post an entry on it before I spill hot coffee all over my notepad and start crying like a little baby. Without further ado, heres my list:

Various Fixes - alt+enter

This tool is equivalent to ctrl+1 in Eclipse and acts as a kind of quick fix tool for you. You'll probably use it the most for the following:

Adding import declarations.
If code is highlighted yellow, IntelliJ is suggesting that you can possibly write your code in a more efficient manner. Simply double click the piece of code and hit alt+enter. This will give you some refactoring options.

Code Complete - ctrl+shift+enter

This nifty shortcut will complete syntactical code for you such as closing braces during a method declaration, or by typing "if" then hitting ctrl+shift+enter will close add the parentheses for evaluation as well as the open and close brackets for the if block. Probably does a lot more but that's what I use this for the most.

Dumb Auto Complete - ctrl+space

This "dumb" autocomplete usually appears automatically after you type the "." after an object variable. For instance assume someArrayList is of type ArrayList and when typing "someArrayList.", suggestions are offered based off of that object appear.

Smart Auto Complete - ctrl+shift+space

This is really useful one. It acts similar to the dumb complete key but there's is a crucial difference. This shortcut narrows suggested methods down by type. For instance, let's take our "someArrayList." example again. Shortly after the period you will get suggested methods based off of the entire ArrayList object. This is the the expected "dumb" auto complete view as I explained in the previous shortcut. However when you instead press ctrl+shift+space, IntelliJ will actually filter those methods based on the type in which the return is being assigned to. So if we were create a integer variable to store the size of someArrayList like so "int size = someArrayList.", smart complete will only expose methods off of the ArrayList object which will only return an int value like size(), hashCode(), lastIndexOf() and indexOf()... Very useful!

View Java Docs - highlight method or class+ctrl+q

Nuff' said.

Highlight instances of a term - highlight item+ctrl+shift+f7

Nifty shortcut that will search, locate and highlight all the matching terms you've selected within a file.

Refactor a piece of code - shift+f6

Highlight code and shift+f6 to view suggested refactoring options.

Extract expression to a method - ctrl+alt+m

There are times where you find yourself writing inline, procedural code that should be in its own method but you just don't get around to extracting it because you're lazy. This awesome shortcut provides a very simple way to highlight code, turn it into a method update your references to it. Just like Geico says, "so easy a caveman could do it". Do it!

Select word the cursor is currently on - ctrl+w

When you really don't feel like using your mouse... geek.

Cycle through open editors - ctrl+left arrow or ctrl+right arrow

Nuff' said.

Cycle through methods in a file -ctrl+up arrow or ctrl+down arrow

Nuff' said.

Once debug servers are setup to launch debug shift+f9+enter

Launches debug server.

Terminate the debug server - ctrl+f2

Nuff' said.

And finally, the usual suspects...

Search for a Java Class - ctrl+n

Search for file in project - ctrl+shift+n

Search for term within project - ctrl+shift+f

Comment out code - highlight lines+ctrl+/

There you have it. Hopefully this can help guide IntelliJ noobs like myself in the right direction. I'm still feeling my way through this tool so if anyone has any other useful tips for IntelliJ, please post a comment. I'd love to hear them!

PEACE.

-Mick

Code coverage & cyclomatic complexity calculations coming to ColdFusion

2010-02-25T14:05:00.000-08:00

We've heard it over and over again that ColdFusion 9 has made excellent headway in terms of providing engineers with great support to execute proper object orientation without all the bloat. Along with CF9 and awesome unit testing tools like MXUnit and Mockbox, unit testing is really becoming a mainstay for a lot of ColdFusion applications and I hope that it continues to stay that way.

However one critical and often overlooked aspect of unit testing is code coverage and cyclomatic complexity calculation. Often times, developers feel that just because they've got a good amount of tests that its "good enough" without taking a step back and asking themselves questions like:

Are my unit tests covering every line of code?
Am I testing for every distinct path of execution inside a method?
Is my code written efficiently or is there room for improvement?

Perhaps you've asked yourself these questions but didn't know what the answers to it are. Well the answers lie in code coverage and cyclomatic complexity...

Code Coverage

Code coverage metrics show you the percentage of code that is covered and not covered by your unit tests. This is extremely beneficial to identifying the "holes" in your unit tests. Uncovered lines means no unit tests exist for those lines which is an indirect measurement of code and test quality. If you have not tested it, then it's at risk for error and should be addressed asap before rolling that beaut into production.

Cyclomatic Complexity

Also known as CCN, it answers the question of "is my code written efficiently?". To boil it down, take a method and imagine all the ways you as the developer can cause simple or complex conditionals. Meaning if's, switches, nested if's, inline method calls to other methods etc...

The basic idea is that each of these "conditional" execution paths add to code complexity. In the programming world you always want to keep code complexity to a minimum because the more complex conditional code you have riddled throughout your code base, the less maintainable it is and extensibility becomes almost impossible. Your software becomes risky as more complex paths are added resulting in unhappy customers and stressed out engineers - and I think you'd agree with me that stress sucks and in a volatile software, release days are nail biters...

So the idea is that you run CCN metrics on your classes and it will spit out a number for you. The standard is that the number should be under 10 and anything over that should be flagged, reviewed and more often that not re-written.

Code Coverage and CCN coming to ColdFusion

In languages like Java and C#, there are tools that provide this level of support but not for ColdFusion. That is until now!

Written with Java and ColdFusion, cfcommon's Chimera project will provide you with that critical line of defense to help engineers identify parts of their code that need unit tests and/or need to be re-factored because of too much complexity.

Attached is a preliminary screenshot of Chimera playing nicely with MXUnit tests.

Although this image shows ColdFusion code writted in tags, we will also be providing script support as well ;)

Please stay tuned as we will be releasing Chimera BETA very soon and you can check out cfCommons in the meantime!

PEACE.

Mick

Building a cfcommons application - Battlefield Part I

2010-02-23T08:51:00.000-08:00

Overview

cfcommon's pluggable architecture allows you to harness many of its stable modules and plugins bit by bit, piece by piece without inheriting bloated code and unnecessary overhead. Each module is carefully crafted and engineered to ensure stability and developed with clear and concise features. Gone will be the days of installing multiple frameworks for different feature sets and having to deal with the baggage of worrying about collisions and incompatibility issues that can occur between them.

To get a full appreciation for what cfcommons is all about, in the next few posts we will create a cfcommons application from the ground up, utilizing each module and plugin "as needed". You will see how incredibly simple it is to not only plug and play features, but also how each module plays nicely with each other.

Without further ado let's create our application using first up cfcommon's Neodymium - an automatic dependency injection plugin.

dependency injection, what is it?

Also known as "Inversion of Control", is one of the most critical pieces in building clean, enterprise level object oriented software. If you are unclear about dependency injection, I encourage you to first read about what it's all about.

Many languages have frameworks that enable your software to leverage dependency injection like Java's Spring to .NET's Castle Windsor to ColdFusion's Lightwire. The old way was to "wire" up these dependencies via XML or bean configuration which is extremely wasteful in terms of keystrokes causing you to run into either a gigantic bean config file or a gigantic XML. A good way to induce coder's coma...

Yea yea, you can break things up into separate XML files or use "shortcut" attributes to help lessen the bloat but still, we're talking about making what will be a thousand line XML template down to hundreds...at least for awhile until it grows even bigger. Thanks, but no thanks..

what do you mean by "automatic" dependency injection?

Neodymium allows you to simply annotate your class properties with the right type and it will automatically inject those properties for you during application startup or reload. Once you ask the factory for a bean it will return to you the object with its dependencies properly injected for you.

so no XML mappings or bean config??!?

None. Nada. Zilch.

what if you don't want objects to automatically be injected?

Then use the exclude annotations.

what if I'm l337 and I want the type to be an interface?

Then use the combination of @type and @implementation, which gives the path to the concrete class. You're so 1337...

Battlefield Application - Part I

Let's create a cfcommons app shall we?! I didn't want to use the played out car/student/shapes/{insert boring, mind numbing tutorial topic here}, so I opted to go with a battlefield app which I'm hoping we can turn it into some sort of army game over the next few posts.

We will start by implementing automatic dependency injection via Neomydium. The battlefield code is also available for you to download and install as you follow along...

install cfcommons and create the web directory for this tutorial

Install the cfcommons core modules library into your web root.
Create a /battlefield/ directory under your web root. This will be the directory that houses our sample application.
Under create a /model/ directory. Here is where we will put all our classes and interfaces.
Under the /model/ directory create an /interfaces/ directory. Self explanatory...

create the context config XML file

This file is the central file that you will drop the cfcommon plugins.

In /battlefield/, create a /config/ directory
Inside the /config/ directory create an XML file called context-config.xml that looks like so:

In this XML you'll see we've declared both a development and production environment and told the Battlefield application that we will use the Neodymium automatic dependency injection plugin via one simple plugin tag.

Once complete, your directory structure should look like so (minus the .settings, that's eclipse stuff.):

ok the good stuff now. let's take a look at what we are about to build.

Simple right? We are going to create a soldier that "has-a" weapon. For all intents and purposes, let's usimagine that by default the Soldier should always be equipped with a knife by default. Meaning every time we need an instance of a Soldier from the bean factory, let's always make sure it has a knife object attached to it.

where the rubber meets the road...

Line 3 baby, it's where the magic happens. By simply annotating the property, Neodymium will register Soldier beans with its dependencies properly injected.

test drive - index.cfm

Here, we are:

Creating an instance of the PluggableContextFactory based off of our context-config.xml we created earlier with the Neodymium plugin installed.
Then we simply ask the context factory for the soldier object.
Then dumping the Soldier and it's weapon.

index.cfm results

Viola! Notice that the Soldier object has a Knife object dependency attached to it. Automatic dependency injection baby...

download the code

Battlefield Part 1 - be sure to also install the latest cfcommons module library here.

conclusion

Hopefully you are starting to see the power behind cfcommons. With one simple declaration in the context-config.xml, we've added dependency injection support in literally a few seconds.

Neodymium also supports the usual suspects when it comes to dependency injection like singletons, interface based typing and custom annotations. Find out more about it here.

In Part II, we will build upon this application and integrate the SimpleMVC then perhaps SimpleSecurity.

Visit http://cfcommons.org/ for more information or to be a contributor.

Till next time, later peeps!

Mick

cfcommons Reflection API

2010-02-19T12:50:00.000-08:00

what is reflection?

Reflection is the ability for a program to introspect details about itself (the source code), understand it and even modify it.

is it possible in ColdFusion?

Yes, don't get it twisted :P

really, how?

In ColdFusion functions like getComponentMetaData() and getMetaData() that will provide you with source level information such as method names, arguments name, type information, annotations, class hierarchy etc. So when you call one of those methods you are essentially using reflection. You are asking the object to reflect upon itself and let you, the developer, know about it.

ok, but how is this powerful?

Simple - let's take cfcommon's lightweight automatic dependency injection plugin called Neodymium. The problem was how do we provide a lightweight automatic dependency injection plugin with these goals in mind:

It must be lighting FAST
Absolutely zero XML
It must be lighting FAST
Absolutely zero bean configuration
It must be lighting FAST
Absolutely no other feature other than dependency injection to bloat the code
It must be lighting FAST

By leveraging cfcommon's reflection API, at application start/reload we are able to to reflect upon all classes (cfc's) in a designated directory (package) check to see that if they have a type attribute associated to it then we will automatically inject those dependencies into the object factory. If there was a doNotInject annotation then we simply do nothing.

See how powerful that is? Extremely. Oh and did we mention it's freaking fast?

if ColdFusion already gives us the ability, why use cfcommons?

Although ColdFusion's reflective functions are helpful, there is a lot of uneccessary code you need to write to get at specific information. cfcommon's Reflection API provides you with object oriented classes that help you get at what you need with one method call. Some examples:

Need to check if a class is annotated? use isAnnotated()
Need to get all annotated methods of a class? use getAnnotatedMethods()
Want to know if a method has an annotation? use hasAnnotation()
Need the superclass? use getSuperClass()

The list goes on and on. You can check out the API source documentation here.

sky's is the limit

The goal of the cfcommons project is to push the limits forward for ColdFusion. It's a call to developers to think outside the box and to help provide stable, reusable modules using a powerful platform. Reflection is just one of many modules available to you.

So give it a try it! Can you think of a problem that you can solve via reflective methods?

i want more information!

A must read about Reflection API WITH examples - reflection documentation.

Questions?

Brian Carr, the mastermind behind cfcommons, can always be reached via:

email : brian@cfcommons.org
twitter: @cfcommons

You can also reach me at micky@cfcommons.org or on twitter (@mickydionisio).

Thanks peeps!

ColdFusion Commons Project Released

2010-02-18T13:04:00.000-08:00

Over the past few months I've been involved with a new project called the ColdFusion Common Modules Project. Inspired by Java's Spring framework, Apache Commons, and ColdFusion's Quicksilver framework. What exactly is it?

ColdFusion Common Modules Project

The ColdFusion Common Modules project (CFCommons) is a collection of initiatives intended on producing reusable, stable, object oriented ColdFusion component libraries.

What's in it for you?

This project is intended to provide ColdFusion developers with highly scalable, reusable components. Components that are created for YOU, by YOU.

You as a CFCommons developer will make an effort to ensure that your components/plugins play nicely with other plugins and that your interfaces are extremely stable and reusable by implementing proper object orientation techniques and interfaces. This ensures that any developer using a CFCommons component can rest easy knowing that installing a new component won't break ten other CFCommon components.

In its first release, CFCommons offers a handful of features and Context plugins that range from simple MVC to layout management, from automatic dependency injection to ReST web services, from templating to service level transactionality. CFCommon's plug and play architecture allows you to produce decoupled plugins by utilizing the CFCommon Context and Reflection API. The sky is the limit on what you can create.

Let's take a look at the CFCommons modules...

CFCommons Modules

Context - The CFCommons Context is a reflective, pluggable application platform. This is the heart of CFCommons.
Reflection - The Reflection API is a small set of components that wrap the standard structure output of the native CF reflection methods, getMetaData() and getComponentMetaData() and provide an object-oriented and enhanced view into ColdFusion component (.cfc) meta-data.
AOP - The AOP module brings full-featured stand-alone Aspect Oriented Programming facilities to ColdFusion with an API that is both easy to understand and implement and that is not dependent upon any other libraries or frameworks.

Leverages ColdFusion dynamic language features to provide a programmatic join point model which supports before, after and around join points.
Fast and easy applicaiton of advice to pointcuts.
Advice types are custom function references, or methods that exist on object instances. No need to have advice implement an interface or extend another AOP API specific type.

Templating - The cfcommons templating module promotes fixed content reuse by providing a means to inject variable content at specified locations within given text. Perfect for the implementation of semi-mutable views such as form-letters or email templates, this module provides an object-oriented set of Components that promote effective templating.
Faceplait - Faceplait is a stand-alone, framework agnosting layout manager.
HTTP- This module provides library components who's purpose it is to make working with common HTTP tasks, easy.
Chimera - A Java/CF based code coverage and cyclomatic complexity tool. (Built to place nicely with MXUnit.) MXUnit peeps, we'll be reaching out soon so stay tuned!

Here's a quick screen of chimera:

Want a test drive? Take Weyland out for a spin.

To get up and running quickly install Weyland along with the CFCommons core. Weyland is a simple HTTP Context with MVC, automatic dependency injection, layout and security support.

Drop the org directory in cfcommons into your web root.
Drop the weyland directory inside of root as well.

The directory structure should look like so:

And after visiting http://localhost/weyland you shuold see this:

From that screen you can try out the secure login example and dive right into the code using the documentation. Download them both here!

Interested in learning more or contributing?

Please visit the CFCommons site, poke around, read some documentation and try it out.

Are you interested in becoming a CFCommons developer? Contact us to submit yourself as a contributor.

Holla.

Immediate Web Performance Enhancements

2010-02-04T16:23:00.000-08:00

Been doing a lot of research lately on web performance and came across the work of the Exceptional Performance Team from Yahoo!. This team is responsible for putting together best practices to achieve high performing web sites.

I've hand picked some of their best practices that are, in my opinion, relatively quick to implement. Below I've listed the tips and a quick summation of each. However I encourage you to take a look at the entire list posted by the performance team. There are a total of 34 best practices spanning 7 categories. I've posted the link at the end of this post.

Without further ado, let's jump right in:

Content

Minimize HTTP Requests

Combine script and css files - During your release process, you can use a number of the tools out there to combine, also known as minifying, your scripts and css files.
Use css sprites - combine images into one file and use the css image styling properties to display the correct image. For instance, if you had a set of images that are essentially rollover images, combine them all into on image so that instead of the browser having to load 10 on/off images, it only needs to load it once in one HTTP request.

Cache AJAX Requests - Expires or Cache-Control headers are your friends. Append a timestamp to your url to ensure so that any subsequent calls to that url will be from the cache. It's important to note that if for some reason the user performs some action that will need to make a request to the url that has already been cached, you need to ensure you update that url's timestamp.

Split Components Across Domains - Per the HTTP/1.1 spec, browsers download two components in parallel per hostname at a time. As a rule of thumb, you should separate your static content like images/css/scripts/documents from the servers that server up your dynamic content. This means that if you have three domains, one that serves dynamic and two that serves static, then you allow 6 "parallel downloads" at a time per request.

CSS

Put style sheets at the top - This tip is more of a pseudo performance boost because the idea is all about progressive loading. It's all about instructing the browser to load items such as images and layouts as soon as possible, once they are available. The HTML page should be your loading indicator.

Choose link over @import - In IE, @import behave just like a link at the bottom of the page which impedes progressive loading. Keep it simple...

Javascript

Make Javascript and CSS external - well it's almost always a good idea. The reason I say almost is that you should be mindful of, as the team puts it, "..the frequency with which external JavaScript and CSS components are cached relative to the number of HTML documents requested.". If there is a page that uses CSS and/or Javascript but that page is only visited 1-2 times upper limit per user session (like the homepage), then it might not make sense to load external files since quantifying the performance boost would be difficult and probably end up being negligible. Better to put it inline so that CSS can be loaded quickly lessening HTTP requests to external files and maximizing progressive loading. Of the course on the opposite side of the spectrum if you have CSS/JS that is being used across different pages and those pages are high trafficed, then you should absolutely place those CSS and Javascript files in external files.

Remove duplicate scripts - no brainer here...

Put Scripts at the Bottom - Putting scripts in the head of the HTML file blocks parallel downloads. If you can get away with it, put your scripts right before the end of the body tag.

Minify Javascript and CSS - Basically this means to remove spaces characters and comments from your JS and CSS files. You can take this a step further and even combine your minified files into one. As with anything in programming, use your discretion as the need for combining scripts or css into one file might not make sense in every application.

Server

Flush the buffer early - If your page is heavy on the back end processing you might consider flushing the buffer. In doing so you will allow you to begin fetching components like images, css, scripts. The ideal place to put this is right after the head tag, before the body.

Add an Expires or a Cache-Control Header - This is inline with their "Cache AJAX requests" rule. The performance team boils it down to two rules.

For static components: implement the "Never Expire" policy by setting far future Expires headers.
For dynamic components: use an appropriate Cache-Control header to help the browser with conditional requests.

So there you have it. Some of the few practices from the Yahoo! Performance Team that I believe you could probably do in a reasonable amount of time.

Also if you haven't already take a peek at YSlow that integrates with Firebug for FireFox. Also Smush.it from Yahoo! is a pretty cool tool for optimizing images without losing quality.

Till next time... PEACE.

-Micky

References

Best Practices for Speeding Up Your Web Site - Yahoo! Exceptional Performance Team

Javascript Closures

2010-02-02T14:47:00.000-08:00

What is a closure?

If a function is created in such a way that it will retain it's local variable scope even after the function exits - then its a closure.

What do I need to understand first about Javascript to understand closures?

Javascript uses lexical scoping. Huh? Simply put, local function variables are kept alive even after the function exits. You'll hear this explained a few different ways - "variables are kept alive after the stack frame/pops off/deallocated/ends/completed/{insert geeky word to signify stack frames ending}".
By default, inner functions have access to variables within their outer functions. It does not matter whether its 1 or 10 levels up. This is absolutely paramount in understanding closures.

How do you create a closure?

Any time you create a function within a function and the inner function refers to an outer function's private variable, you've created a closure. Simple as that, don't over complicate it.

A Simple Closure

In this example, here is what's going on...

Line 1 - We create a function called multiplyTemplate with one argument.
Line 2 - The argument is saved to a private variable called multiplier.
Line 3 - We declared an inner function. Something should have clicked here. Remember what I mentioned about inner functions having access to parent function's variables scope. multiplyTemplate is now a closure. Why? Because it is using its PARENT function's private variable multiplier. What's going on here to the end of this inner function on line 5 is absolutely critical to understanding closures.
Line 4 - We return the value of multiplier times y. Multiplier is a private method of the outer function.
Line 8 - we create a new variable that executes multiplyTemplate(5). Remember the return of is a multiplyTemplate function, not a primitive int value. It's on this line where the rubber meets the road. Here is what just happened:

multiplyTemplate(5) was fired and a new function context was created.
The argument of 5 was assigned to this function context's private multiplier field.
An anonymous function was returned whose value of x is being RETAINED/KEPT ALIVE even though multiplyTemplate(5) has exited.

Line 10 - multiplyBy5(10) is executed using the function context declared on Line 8. The argument 10 is actually the y argument on line 3 which uses x to multiply to itself. The multiplier x is still 5 through closure and the value that returns is 50.

Why use a closure?
I'm sure there are a handful of reasons where you would want to use a closure, however in my experience I've only used them for a couple reasons.

On main reason is if I need to properly encapsulate data inside an object whether it be transient or a singleton.
Attaching event handlers to a DOM object via anonymous methods+closure inside of a for loop.

That's the skinny. There are many many many resources out there that explain closures in less-than-understandable format but hopefully this helps demystify it a bit.

-Mick

The Unit of Work Design Pattern

2010-01-18T13:51:00.000-08:00

"Everything that we do in this grid or in these fields, or in this group of tabs cannot be saved to the system until the Save button is actually pressed", says the business analysts.

This made Brian Carr and myself sit way back in our chairs and go "Hmmm... How can we solution this problem given our architecture?". The piece of software we're developing is built around SOA - our front end user interface is strictly ExtJS that communicates over ReSTful web services to our platform. Not a lick of CF on the front end. Shortly after those requirement gathering meetings we buckled down to analyze the problem at hand.

Here were some of our main considerations:

ExtJS grids come with their own dirty/clean mechanism but that only existed for grids. Our client's requirement existed for any form control, not just grids. That option was out of the picture, especially in a SOA.
Even if that functionality existed for all form controls, how would the platform handle saving these data points simplistically with minimal overhead and a clean API? In a traditional application that doesn't use data mappers or ORM, you'd usually have a post page that inserted/saved everything about that entity - for instance a User. You'd have a web form that when posted, would update all 20 fields even though all you needed to update was one email address. In an enterprise level environment this is a HUGE waste both in sending data over the pipe as well as causing unnecessary database overhead.
We already had our own ActiveRecord implementation in place so handling crud operations from our domain model was extremely simple.
We already implemented the Observer Pattern that I've detail here, to handle saving composites among other things.
At a systems level, we wanted to make small, light and quick web service requests to our platform whenever a form field was updated.
It must be transactional.

We knew that when it came to persisting an object singularly, it was solid. The api was such that these objects could save itself, load itself etc. We also knew that via our Observer Pattern implementation that when an object saved itself and there were any composite relationships attached to that instance, any of those composites would also save itself. So the only two problems where this:

How do we logically group disparate objects in memory that are part of a unit of work - like a front end data grid or a group of data separated by tabs - to ensure that the underlying objects that encapsulated these data points can be saved, deleted or rollbacked (removed from memory) when needed. All while using our ActiveRecord and Observer patterns?
ALL of this must be managed across multiple independent http requests to the platform.

As usual Brian and I banged our heads against the wall trying to come up with a solid solution. Then it happened... enter the Unit of Work design pattern by Martin Fowler.

Maintains a list of objects affected by a business transaction and coordinates the writing out of changes and the resolution of concurrency problems.

Let's jump right in. I won't go over the exact implementation that we used in our software, but I'll provide a quick overview of this design pattern and how you might get started with it.

As always, here is the class diagram first.

So you'll see here that we've got:

An IPersistent interface that has save and delete that returns a string for example purposes.
A UnitOfWork (UoW) object.

that has 3 fields to store the new, delete and changed stack. Which all should take an array of objects, preferably strongly typed if your language can support it.
methods for adding to each stack.
a method for committing and deleting the UoW.
a get() method that will pull out an object from the UoW registry.

The idea is simple really. The UoW acts as a container/wrapper for disparate objects that will need to take part in a single logical transaction.

Here is the code for the UnitOfWork object:

Pretty straight forward here. Class members are created to hold objects (stack) and methods are there to throw objects onto a stack. Notice how the entirety of commit() is transactional - it is all or nothing. We iterate over all the stacks and call the appropriate crud method. Also the get() method will pull out the object from the registry based on hashcode. You can do this however based on how you uniquely identify objects in your system.

Here's the scratch code:

Which produces:

In this test we:

created three users
displayed the unique hashcodes for each user
assign user1 and user2 to to the UoW new stack.
assign user3 to the UoW delete stack.
then call commit() on the UoW.

Cool? On uow.commit() we've executed the appropriate database method to handle persistence and printed the save/delete messages to the screen. Of course in a real implementation this would be whatever persistence mechanism you have in place - DAO, ActiveRecord, data mappers, whatever.

Here are a couple key things to consider that this post didn't cover:

You will most likely need a singleton object, usually called the UnitOfWorkManager, that is responsible for managing your UnitOfWork objects. This plays into the feature of supporting UoW across multiple HTTP requests.
You will need a method on the UoW object that will remove objects from certain stacks altogether. This is for situations that perhaps the user clicks a "new user" object and a new Use robjects get registered. However he/she decides that a new user is no longer needed so they subsequently remove it but there are still other data changes that need to be persisted for that UoW. Don't get this confused for rollback() or registerDeleted().

This is one of those design patterns that you can find yourself getting really deep and start customizing like crazy. Be careful for that, make sure to keep it simple...

Hopefully you can use this as your jumping off point... Happy UnitOfWorking!

-Micky

References

Martin Fowler's UnitOfWork

My Brother's Term Paper - Strengths and Weaknesses of Object-Oriented and Procedural Programming

2010-01-13T06:56:00.000-08:00

My 17 year old little brother, Gian Dionisio, is currently working on getting into college to pursue a computer science degree and has always expressed a genuine interest in software engineering. I'm on his back 24/7 when it comes to good grades, school etc. so it was a pleasant holiday present when he showed me his senior term paper in which he received a 98 out of 100. Glad he hasn't caught "senioritus" like his older brother did! I'm posting this more for selfish reasons as it's something I'd like to point people to for a totally objective view when it comes to object oriented vs. procedural programming. I've probably given Gian about a 1 minute dump of my own thoughts on the subject but as you'll read, his essay is based entirely on his own research. so here it is, good work Gian, very good work.

Strengths and Weaknesses of Object-Oriented and Procedural Programming by Gian Dionisio

The process of designing a program is broken down into four steps: analysis, design, implementation, and maintenance. Analysis refers to determining if a problem is solvable, design refers to the process of determining how to solve the problem, implementation is the actual coding of the program, and maintenance refers to the debugging and modification of the program. As the process progresses from one phase to the next, costs rise higher and higher, with maintenance being the most expensive (Lambert). Currently, many programmers use one of two paradigms of programming when designing and implementing a program: procedural and object-oriented. However, many programmers have begun to stray away from procedural programming in favor of object-oriented

because of the way object-oriented programs handle data, which allows for more dynamic and flexible programs. Even though object-oriented programming offers more flexibility than procedural programming, programmers should not perceive procedural programming as obsolete because both paradigms have strengths and weaknesses that do not make one necessarily better than the other.

Procedural programming is best described as a one-way system, using only variables and functions, with no major data grouping, which is the primary strength of object-oriented programming (gabehabe). This paradigm emphasizes on conciseness and reaching the end result as quickly as possible, solving only the problem presented and nothing more, making procedural programs very straightforward (Weisfield). John Barton of IBM notes that generations of programmers have essentially been taught to create a list of instructions for the

machine on how to read data (gather ingredients), how to use equations and functions with subroutines (cook the meal), and how to display the product (a finished meal) (Waldrop).

While the design and implementation of a procedural program may be straightforward, especially in smaller scale projects, as the programs grows larger in scale, both design and implementation become increasingly more difficult as different subroutines are being performed on the same set of data at the same time (Waldrop). When the program is finally functional, modification is made either difficult or impossible because changes in one part of the program may affect the rest of the program as a whole (Archwing). Because procedural programs are not able to be readily modified, maintenance costs become very high. For example, think of a 10-story apartment complex with 10 apartments on each floor, and a development lot for 100 houses; the lots of houses may cost more to construct than the apartment complex, but the houses are, for the most part, self-contained with their own

utilities and walls, versus an apartment, which shares utilities and walls, therefore adding another house would be far less expensive than adding another apartment to an apartment complex (Orr). This problem of modification is solved with the principles emphasized in object-oriented programming.

The main strength of object-oriented programming is its ability to handle data by taking advantage of the way we perceive of environment with the use of objects (Parr). Objects can be seen as separate entities, each with their own characteristics. These entities all share two characteristics: states and behaviors (Sun). For example, a

dog has 'states' (name, color, breed) and 'behaviors' (barking, fetching) (Sun). Objects "encapsulate" these qualities, meaning that they contain data, otherwise known as states, and methods, otherwise known as behaviors (Orr). These objects inherit these qualities from blocks of code known as "classes"; a class can be seen as a blueprint or template of an object, much like how biologists group organisms into classes such as "bird" or ”mammal” (Waldrop). This use of objects gives a solution of designing large scale programs by writing small-scale software units that communicate with each other, rather than a huge system (Parr). This tactic of "divide and conquer” is employed in both procedural, with the use of subroutines, and object-oriented programs, with the use of objects (Nemirovsky). However, in a procedural program, the programmer must tell the computer how to perform an action and when to execute it, in contrast to an object-oriented program, where the programmer only tells the object what to do, and then the object performs it (Sherer). This means that a programmer doesn't need to know how or why an object works, but merely what it does; demonstrating the principle of information hiding, a method of encapsulation in which only certain details are able to be seen (Orr). Information hiding allows for large scale projects to run much more smoothly because programmers don’t need to know the implementation of a class to use it, and allows changes to be made without drastically affecting the entire program, as well as a certain degree of consistency within the application, as programmers working on the project are using the same base classes

(Lindsay).

When code is segmented into smaller sections, each with their own separate individual functions, programs become easier to work with. For example, in an object-oriented program, if a program needs to know the month in a two-digit format, it sends a message to a Date object with the message asking for the date in that format, rather than accessing the date from a variable floating somewhere within the program (Waldrop). Within that same program, another object can ask for the date in a three-letter format by sending a message that corresponds to that demand to the same Date object, and the Date object would return that date. Object-orientation also allows the developer to assemble his or her code in the same manner that he or she thinks about the problem (Waldrop). The developer can create an object named ”Airfoil” and define a function that instructs the object how to handle data, and then the developer can send a message to the Airfoil object to calculate something, such as wind resistance (Chastain). By developing and debugging small components independent from the program, developers are able to isolate and test code more efficiently, and can assume that the program works as advertised, rather than guess where problem may lie (as with the case of a procedural program) (Chastain). By breaking up code into smaller pieces, object-oriented programs become easier to modify, and because of this many companies are interested in reaping the benefits of the object-oriented approach, that is, reusable and easily modifiable programs (Patrizio). Object-oriented programming can be considered better at representing the real world than procedural programming because object-orientation allows for more intricate and dynamic interactions as well as allowing non-technical workers to better understand and participate in the maintenance of a program because objects better appeal to the natural human cognition patterns (Archwing). An example of an object-oriented program’s ability to be modified can be demonstrated in a payroll program; if the program was written procedurally, the area that assesses the employee's paycheck would be in an "if-then" form, versus an object-oriented program, which would send messages to all the employee objects, which would then calculate all the paychecks (Sherer). Any modifications done to the procedural program might cause a chain effect in the program, making unintentional changes. Object-oriented programs also become easy to build upon in case the scope of the project grows larger

because of the principles stressed in object-oriented programming, such as encapsulation and inheritance, allow the application to expand quickly, and pre-existing classes do not have to be modified in any way because objects interact with each other through their methods and messages (Chastain).

Object-orientation offers many tools and principles that allow programmers to create dynamic programs; however, learning to use these tools can prove difficult. An entire way of thinking was built off of the procedural method, and “reprogramming” the programmers may take a lot of effort on the developer’s part before they can fully reap the

benefits of clarity and reusability (Waldrop). Procedural programming is action-oriented and solves problems through a series of logical steps, versus object-oriented programming, which looks at the entire problem as a whole and then derives a solution with the use of a series of reusable classes (Patrizio). The object-oriented approach presents a radically different method of method approaching software: code and data become merged into one single, cohesive unit— the object, and is an abstraction of a set of real-world things, such as "date" or "employee" (Archwing). Systems are made up of multiple objects, such as date, or processing, and make requests with messages which ask for specific pieces of information (Archwing). This different way of thinking can be met with resistance by some, as in the case or Chevron’s programming staff, who have difficulty grasping the concepts (Moser).

A method for tackling the problem of learning how to use object-oriented programming and concepts is through the use of graphical user interfaces (Moser). Graphical user environments have been in use by high school computer classes in hopes to intrigue students to Iearn object-oriented programming by showing the fun, enjoyable side of programming without having to touch the dense terminology of the industry (Demski). For example, many high schools have begun to use the BlueJ graphical user interface, which displays coded boxes and arrows that allows students to watch the concepts in action, and allows them to get a clear understanding of the concept (Dernski).

However, learning how to use object-oriented programming is not the only problem, but when to use it. Object-oriented programming, while very good at managing data complexity, is difficult to apply, which may be a major reason for programmers not to adopt object-oriented programming. Marc Funaro, a computer programmer, made a blog post detailing his experience in trying to incorporate object-oriented programming principles into his own programs, and stated that by doing so, had almost ruined his business (Funaro). Marc was under the impression that by incorporating aspects of object-oriented programming, his programs would become much more efficient and reusable (Funaro). However, this is only partly true, because in order to fully take advantage of object-orientation, one needs to learn how use object-oriented analysis and design; using an object-oriented language will not yield the promised benefits unless one learns to think in the appropriate manner (Buckier). Brian Carr, a seasoned software architect, stated that programmers who use an object-oriented language but are not thinking in an object-oriented way are essentially procedural programmers; additionally, learning how to use object-oriented programming does not make one a better programmer, but rather, adds another tool to the programmer's disposal

(CF OOP Debate). Carr also notes that object-oriented programming is better than procedural in one aspect: managing complexity. Funaro also complained about design and implementation time when using object-oriented programming, and that a project done procedurally would be completed much more quickly (Funaro). Mike Chandler, another object-oriented programmer, commented on this, saying, “There are some projects that procedural programming would be better suited than OOP, especially those that are smaller scale. When you’re designing a program, you really need to think about the customer’s needs. Will he wants to modify or extend the program? That’s when you need OOP” (CFOOP Debate).

In this age of technology, advancements are being made very quickly, however just because progress is being made, doesn’t mean we should abandon older methods. Object-oriented programming, though powerful and very flexible, is not without its drawbacks, suffering from longer design and implementation times. Procedural programs, though inflexible, are quick to design and implement, and may even prove superior to object-oriented programming is smaller scale projects if fast delivery is key. However for the mid to enterprise level software systems, the object oriented approach does cost a little bit more up front but extraordinary gains are realized during the maintenance and enhancement phase, which is where most cost is incurred.

Fundamental Lightwire Bug?

2010-01-08T10:48:00.000-08:00

**UPDATE 01/09/10** Peter Bell, the creator of Lightwire, has addressed this issue. You can find his blog post about it here. I've also update this post as well.

This bug seems extremely fundamental to be a real bug so my initial thoughts are that I'm doing something completely wrong. An issue like this would have been discovered by now.

Brian Carr and I were working on some front end development for our project that leverages Quicksilver and Lightwire when we came across an bug in the UI. Whenever a form was submitted that contained form validation errors, it appeared to be returning correctly the first time. However if you were to submit the same form again, we noticed the error messages would continue to duplicate and stack. Initially we thought there was an issue with Quicksilver not configuring the beans correctly or perhaps we tagged our Errors object with a singleton annotation. After some digging into our bean configuration annotations, everything was annotated correctly.

So next step was to switch our dependency injection provider to ColdSpring, which Quicksilver allows you to do with one line. What happened to our surprise? It worked! The errors were displaying correctly. Ok, so now we know that the issue is Lightwire specific but didn't know whether it was a Quicksilver or Lightwire problem so we dug into Quicksilver's injection provider service for Lightwire. Perhaps we're wiring something incorrectly. After a good amount of investigation we verified Quicksilver was setting configuring beans correctly for Lightwire consumption.

Our next step was to completely bypass Quicksilver altogether. If we can configure these beans using just Lightwire and a scratch pad and still seeing this error then we can safely deduce that it was a problem in Lightwire. So here is what we found...

First, here is the jist of the classes involved. Both are TRANSIENT objects.

Simply, a BusinessServiceResponse has an Error object. For this example, the hashcode member is a unique hash that is attached to each unique instance.

Here is the code that we are using to test:

Line 4 - 14 Configure the two transients - Errors and BusinessServiceResponse
Line 16-20 Configure setter dependency. BusinessServiceResponse has an Errors property to be injected with the Errors transient object.
Line 24-38 On the first iteration we are getting the bean for the first time then displaying the hashcode on the BusinessServiceResponse and the Errors. It's important to note that Errors hashcode is accessed through the BusinessServiceResponse on line 37.

Here are the results:

It appears that errors is being handled as a transient as the hashcode are identical across all three BusinessServiceResponse creations. Now that we know what the problem was, we dug into the Lightwire code. We found the issue in Lightwire.cfc. It loads setter dependencies as singletons by default??? Nooo, can't be.

On line 273, its creating each bean as a singleton. After replacing line 273 with an singleton/transient check like so:

It worked. Let's rerun our tests.

Hashcodes are different meaning now they are transients as well.

*** UPDATE 01/09/10 *** Do NOT use the code I posted as a fix as there has since been an official patch to fix this issue. Grab it from here. Many thanks to Peter Bell and Brian Rinaldi for their help and quick response on this!

Please guys, help me see the light...

Quicksilver - REST, MVC, AOP in 26 lines of code, 6 annotations and 0 xml.

2009-12-17T12:07:00.000-08:00

You read it right. Let's jump right into how the Quicksilver Framework accomplishes this.

Here is the view - getStuff.cfm

And finally, the results when run:

So, how did we just implement MVC?

STEP 1 - Annotate your component with @viewRoot. This is where all your views are stored.
STEP 2 - Map the uri and http method to your url. You can optionally use the @returnTypeName annotation so when defined, it's value is mapped to the methods return and will be available in the request scope of your view Look at the getStuff.cfm screen cap above..
STEP 3, non-step - Sit back and relax. Watch the goodness of convention over configuration. QS will look in your @viewRoot for a directory named after the controller, in this case SampleService. Then within that directory look for a file whose name matches the method being mapped.

How did we implement AOP?

STEP 1 - Annotate your cross cutting method with @aspect and give it a name.
STEP 2 - Decorate the target method with @preAspects and/or @postAspects annotations. A comma separated list of the aspect names.

So when getStuff() was run, Quicksilver first ran the aspect addRandomStuff(). All in 2, dos, ni steps. Cool eh?

How did we implement REST?

NO STEPS! - You already did when you annotated for MVC.Proof? Lets hit that URL again but leverage Quicksilver's automatic response serialization (of array, structs, queries) and get back a JSON representation of "stuff" instead of going through MVC. Simply append ".json" to the end of the url, nothing else needs to change. Hit the url with .json and you'll see this:

That's all folks. All of this in 26 lines of code using 6 annotations and 0 xml.

Hope this helped and please hit us up with any questions.

-Micky

Design Before You Develop - Intro to Class Diagrams with Quick Reference

2009-12-15T09:10:00.000-08:00

If you already are a design-before-you-develop junkie like I am, daps to you. You can probably stop reading here and move onto the next blog in your queue. For the rest that are curious about it but don't know where to start, read on. It's my opinion that you should get into the good habit of designing before you develop. Even if the design takes you 10 minutes on a scratch pad, a quick design session on a whiteboard or several days in a full blown modeling tool, you should do it.

Why? Because it's during those sessions that you and your team can express the intended inner workings of your software modules, poke holes in it, rinse then repeat as needed. You can focus on a list business of requirements, design the class diagrams and literally step through the requirements allowing you to articulate the path of execution that'll take place to fulfill each requirement. All of this is done without a lick of code. You discover design flaws early on, not during development. That's risky. You can even take it a step further and generate sequence diagrams against your UML model and requirements, they are invaluable! But sequence diagrams are outside the scope of this article, we'll get to it some other time.

I'll say it again and again that during the design process you need to forget the code. Coding should be the easy part. It's the design that lends your software scalability and flexibility. Without a proper design, you run the risk of causing not only future headaches for yourself and your team but more importantly, the business. Yea yea, without the design phase it's so much faster to release a product. Sure, you can tell your boss/client what you're doing is rapid development. Sure, your bosses will be happy for the time being cause you got it out so quick. You're the hero. You're the man. But who cares if you are spending the next year after your go live fixing code, trying to re-design after the fact and burning through company cash hand over fist. Or worse, bandaiding already production level code with cut rate code because everything is an "emergency" and you don't have the luxury of thinking things through anymore. All the while the boss that was singing your praises is no longer impressed that you put out a piece of software that constantly breaks. It's a vicious cycle and could've been avoided by dedicating a little more time to designing up front. Use the above as a case to your boss when you need a little more leeway to get some design cycles in in the beginning. Just like Ben Franklin put it:

An ounce of prevention is worth a pound of cure.

He was a pretty smart dude, and his quote certainly applies to software engineering. Let's put in that ounce of prevention by designing first eh?

UML to the beginner can seem pretty damn intimidating. In my comp sci class I remember cracking open the UML section and my eyes glazed over at all the fricking symbols. I thought to myself, how in the hell am I going to memorize all this? After I got some real world experience, it's really not that bad. Given, you can get pretty deep into it I find that I'm able to express a software model with only a few symbols. I thought I'd share with you some of the ones that I use all the time with Microsoft Visio however the symbols below are not application specific, they are universal symbols. Here's the quick reference:

These symbols should be more than enough to get you going. Below is a what a real class diagram might look like using all above symbols:

In the above class diagram, there are two applications being modeled at the same time that use shared components. A school app and a war game app. I've put together some questions I'd ask myself while making some observations of this class diagram:

Person is being extended by Student and Soldier.
Soldier is in the common package. "Is this correct? Shouldn't it be in the wargame package? What happens if its not?"
Soldier can have many pistols. "But is that true? Should it be a one to one? Should there be a collection manager of sorts to keep the api simple? Better ask the business."
Student can have many books via its multiplicity. "Same multiplicity questions apply here?"
Student is teachable, since it implements ITeachable. "Are the methods defined here really granular? Do they really belong in this interface? Is there anything else I can add to this interface?"
A pistol implements IWeapon so it must fire.
fire() on the IWeapon interface returns numeric. "Is this the proper return type?", "Should it be another return type?", "Are we going to calculate fire() based on the numeric damage its dealt?, If so keep it. If not, let's redesign".
Soldier doesn't have an interface. "Does Soldier need an interface?", "Is there something that all soldiers must be able to do? Let me dig into the requirements and do some textual analysis."
Person has a firstName, lastName and age member. "Are these shared across all Person objects?". "Do they need to be specific to a subclass?"
All classes are have their packages before the name so it's clear where each class should exist.

This list could go on and on!

All of these questions should be addressed during design to ensure you and your team have a clear path of development. There are a bunch of open source tools out there especially in C# built into Visual Studio (and I'm sure Java has theirs too) that will parse through class files to generate UML diagrams ad hoc. For ColdFusion developers, at the time of this entry, Mark Mandel has been recently tweeting about a prototype he's working on that will generate UML using ColdDoc and UML2 tools. Initial screen caps looks promising, so it's definitely something look out for. For more info check out Mark's twitter.

I'll also add that it's important to note that these tools should not be misused. Meaning you shouldn't develop first just to produce a class diagram against off-the-cuff code. These tools should be used when you need a way to dynamically create accurate class diagrams from current code for a number of reasons:

You may be working within an enterprise level organization where there are multiple teams with their own special access to certain aspects of the code. Developers often need to know how to leverage an API in which they have no access to the source. In my experience that's where these automated UML tools come in handy. You can print it out and ship it off to whatever team needs it. From the UML, they can discern the contracts in which they need to develop against.
You don't have to deal with the headache of documentation maintenance.
You need to reverse engineer and analyze design at a high level, especially if the code base is pretty huge and there's a lot going on.

So that's class diagrams in a nutshell and hopefully you'll give the design before you develop approach a try if you haven't already. It may seem like a long and drawn out process at first but believe me practice makes perfect. Take it little by little. Maybe try it with that new module that needs developing or that module that is in need of some major refactoring that you've been putting off for months. After awhile, you'll be zipping through design sessions quickly because you've trained yourself to identify patterns and problems during that analysis phase. It'll eventually become second nature and you'll soon feel dirty not doing some form of design up front.

Hope this helps.

-Micky

Resources
Beginning C# Objects: From Concepts to Code - HIGHLY recommend this book for the beginner UML/OO developer. C# is in the title but the content applies to all languages.

Quicksilver and REST

2009-12-13T12:30:00.000-08:00

Let's talk about some REST a la Quicksilver shall we? In my Quicksilver primer post I touched on what it's all about and its automatic dependency injection capabilities. Today we touch on another one of its core features - REST. If you are not familiar with the concept I encourage you to look into first before reading this post.

Let's set the stage. Imagine you built a simple informational design patterns site that explains design patterns. It's been in production for awhile now using OO best practices like service objects to be the intermediary between internal api calls and the data layer. This site has been using Quicksilver because they wanted a powerful flexible framework to one day use AOP and REST in addition to Automatic Dependency Injection just in case they needed to open up their API to other programs. Fast forward to now, they've been getting requests to open up their API via web services. They can use Quicksilver's REST support to do this easily.

To understand how QS provides a simple solution to this, let's explore a possible alternative. They could develop soap based web services that consumes their service layer. Not a bad approach, but possibly time consuming and if done incorrectly can also lead to duplicate code since the methods will need to understand how to return json, xml, or a custom format. Also each web service endpoint would be exposed through a cfc - /path/to/webservice/DesignPatterns.cfc?wsdl and call methods off that. Ugly ugly ugly. Doing it this way is unorganized and resources are all over the place with crud-type methods names - saveThis(), updateThat(), getThis() etc. Get my drift? How does Quicksilver solve this?

Let's dig into the code that we are about to REST'ify.

We've got a design pattern service with one method getCreationalPatterns(), it simply returns an array of strings whose values are pattern names. Here's the code:

Simple enough right?

We've got a singleton annotation at the top that tells Quicksilver that this should be handled as a singleton. This is part of automatic dependency injection baby.
getCreationalPatterns() simply returns an array. In the real world this could (and should) get as complex as you wanted to, perhaps delegating to a DAO or gateway or even another service but for this example let's KISS.

Good, now take a look at the unit test written for this. Again keeping it simple, let's assert that we are getting five elements back. Using MXUnit, let's write it:

Test passed. Sweet.

Ok so let's now expose this method as a RESTful endpoint that returns a json representation of creational patterns shall we? Let's not create a new component and go through soap based web services, it too much work :) Let's use Quicksilver. How? Use two, count em, two simple annotations in your service object.

@url - this annotation is used to specify your url pattern.
@httpMethod - possible values are GET, POST, DELETE, PUT. You can use multiple methods separated by commas.

So here's the updated code:

Lines 8-11 baby, that's all you need. Now when I crack open my browser and hit:

http://localhost:8500/learnqs/howtorest/api.cfm/design/patterns/creational.json

We get a json representation like so:

When opened looks like:

See how easy that was? We've just opened up our API via restful web services to our consumers quickly and easily.When this URI is called from an ajax request, consumers are free to use the json representation of the creational patterns however they wish.

Lets take a closer look at that URI:

Notice:

We've mapped an HTTP GET call to the url "/design/patterns/creational". Talk this out. Map a HTTP GET to the /design/patterns/creational resource and give it to me in json.
@httpMethod GET is the annotation to map an http method a function
api.cfm is the required cfm template that must exist for CF to parse the uri. This can be named whatever you want. index.cfm, rest.cfm, I just arbitrarily named it api.cfm. Again, as with any request that must go ColdFusion, you must go through a cfm page. In Quicksilver, however, that cfm page does not need to exist. Just the fact that its in the URL is enough, we'll take care of the rest. In our example, api.cfm is not a real file.
After api.cfm comes the path we set in our @url annotation.
Automatic Serialization - Quicksilver will automatically serialize arrays, structs and queries to json for you and since our return type is an array, automatic serialization is kicks in.
By default the return of this call also returns an HTTP status code of 200. You have full control over the status codes as well.

Easy right? Imagine this uri being called via ajax call from jQuery, ExtJS or whatever ajax feature you use. You'll get a json representation back.

Lastly, you've heard use say again and again that the Quicksilver framework does not get in your way and keeps your codebase HIGHLY testable. We're not just saying this to sound cool, we'll prove it. Lets re-run our unit test even after we activated REST.

Look at that :) Test still passed. There's your proof. Quicksilver doesn't inject itself into your code rather it hangs out outside of it. It allows you to implement code and test without the framework in your way and we mean it. How? Via annotations baby! Read more about in the Quicksilver primer post if your confused.

side note: Just FYI - Quicksilver also supports custom representations prepended with whatever extension you want - .extjs? .pdf? .myCoolExtension?. Also allows you to have FULL control on HTTP status codes that are returned as well as the ability to send variables via the URL, http raw post, form post etc. No problem. We'll get to that in future blog posts, promise. Or check out Brian's Quicksilver blog. Link at the end of the article.

So to wrap up, I hope you now see how Quicksilver is as unobtrusive as possible. We REST'd our application with no added components, no messy soap web services, no duplicate code all while remaining organized and highly testable. No framework artifacts riddling your awesome code. How about that? A framework that actually cares about risk management.

-Micky

Resources
Quicksilver Blog - entries by the one and only Brian Carr. Check it out!

Quick Tip - See nested conditionals? Consider Guard Clauses

2009-12-11T14:22:00.000-08:00

Just a quick tip when you are refactoring code with crazy nested if/else logic. As a rule of thumb if you see a method that has conditional statements check to see if these exists:

Is there a private variable, let's call it x that is private to the function that is being set within each (or at least a majority) of the if/case legs whose value is a method call?
Is x is being returned at the end of the method?
You got nested conditionals?

Here is what it would would like:

Notice the:

result private variable at the beginning of the function
result is being returned at the end of the function
Nasty nasty nasty nested conditional that sets result to the value of a method

If the above is true or the code you are refactoring looks similar to this, you might want to look into using guard clauses. Sounds more complicated than it really is. Basically see if you can refactor it like so:

Remove the private variable thats being set during each if/else leg.
Return the method or the condition that is setting it.

Simple as that. Here is the code above refactored:

Notice now:

No more nested if's
After each if/else leg, we are returning the method.

Given that some legacy methods you are trying to refactor are not as trivial as this and each have their own unique level of suck - this is always a good start. This keeps your method simple, less complex and highly testable because each execution path is linear. Guard clauses baby, a good alternative to the strategy pattern.

HTH

-Mick

Understanding Strategy Pattern and Polymorphism - Two Birds With One Stone

2009-12-10T08:48:00.000-08:00

First off, many thanks to the people that messaged me saying they enjoyed the Observer Pattern tutorial for saving composites. I'm really glad it helped demystify patterns and OO for you! And can I add it was pretty cool that these people were a mix from both CF and C# camps :) I am a big advocate of design patterns so I will not be shutting up about them anytime soon, sorry!

Today I wanted to touch on my favorite behavioral pattern - the Strategy Pattern.

What does this pattern solve? If you have an object that:

Is constantly being subclassed just to override specific implementations (methods).
Has a fugly method(s) that has to run through a mine field of if/case statements just to execute.

Let me start of by first saying when you use this pattern you can truly see polymorphism at its finest. And yes, polymorphism is achievable through ColdFusion. If someone tells you different, they are smoking crack. Say no to drugs.

I won't bore you with the computer science explanation of polymorphism so here's my simple, to-the-point summation. If objects are of different types but share a common derivative(s), they should be interchangable. This should help, lets keep it simple:

If A = B and A = C then B = C.

In the example above, B and C can behave polymorphically because they both share a commonality - they both are a derivation/implementation of A. Understand this critical principle of OO and you are well on your way to your OO awakening.

Moving on. Here's the official definition of the Strategy Pattern:

Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it.

What's this mean? I've underlined the key words in this definition. When you hear algorithms, think verbs which mean methods. When you hear encapsulate, think about delegating responsibilities to new objects whose sole purpose is to protect "something" and ensure that these objects are doing only "something" and doing it well (SRP principle). And lastly, when you hear interchangeable think polymorphism. I'll reword that definition a bit to be in line with what I just described.

Define a family of methods and make sure you've encapsulated each distinct method implementation (algorithm) away into new objects so these objects may be switched out polymorphically at run time.

First, here's an abstract class diagram I put together:

For the purposes of this tuorial just imagine that the inside of the legacy code for subject.doStrategy() you've got the if/case problem going on. It's riddled with them.

e.g. -if something, set this then return this way, if something else set that and return that way. You get the idea.

In the class diagram above notice:

We have a Subject, think of this as concrete class that has a composite relationship to an implementation of IStrategyBehavior which subject.doStrategy() delegates to.
The idea is that subject.doStrategy() should NOT have any switches or ifs or overridden via subclassing just to provide concrete implementation.
Also notice the UML note. We delegate to another object that implements IStrategyBehavior through composition (Subject is composed of IStrategyBehavior), this is where that whole "encapsulate each one, and make them interchangeable" is happening. We're saying listen, Subject, yes you will doStrategy() however you are delegating to a "strategy" property and ask it to call its own doStrategy(). This means at runtime you, Subject, don't care about how your are doStrategy()'ing, you're handing it of to your polymorphic property "strategy" and calling doStrategy() on that. We've just encapsulated that behavior away from Subject into its own set of classes or family of algorithms.

OK now we can move on to concrete designing but first lets understand our requirements We're going to use the standard gaming example well because COD:Modern Warfare 2 just came out and it's glorious. (xbox live : Error401, add me)

Build a Soldier that can fight()
fight() cannot have any if's/switches inside of it
fight() will execute differently based off what weapon Soldier is holding - Pistol, Rocket Launcher and Machine Gun

Here is the concrete design:

Here notice the following:

We've modeled a Soldier (Subject) and are creating a strategy pattern for subject.fight().
Interface ISolder has a fight() method with the same contract for for fight() in IFightBehavior.
We've created three concrete implementations (concrete strategies) of IFightBehavior - PistolBehavior, MachinegunBehavior and RocketLauncherBehavior.
Side note : This pattern can also be achieved through an abstract class and subclassing but try to stay away from inheritance as much as you can. Favor composition and interface based programming. You have more flexibility that way.

Code time - Soldier.cfc (I've omitted showing the interfaces. It should be straight forward how to create them from the class diagram above):

Critical piece of code on line 2 - fightBehavior is polymorphic. Interface based programming baby. In most of my examples I will program to an interface, not an implementation (concrete type). At run time this property can be switched out by any object that implements IFightBehavior. That's polymorphism peeps. Simple as that. Don't over complicate it.
fight() delegates to the fightBehavior property. In the abstract portion of this entry, when I said that a Subject, in this case Soldier, doesn't care how it does doStrategy(), in this case fight() - this is exactly what I mean. You're simply calling getFightBehavior().fight(). Soldier doesn't care how its actually doing it because it could be doing it n number of ways through polymorphism. All Soldier cares about is that fight() returns a string. And we the developer have ensured that it will since fightBehavior implements the IFightBehavior interface which also has a matching fight() method that expects a return of string.

Here is what your concrete behaviors might look like. I've consolidated it into one image in order - PistolBehavior (goes Bang!), MachineGunBehavior(lots of bangs!) and RocketLauncherBehavior(goes Boom!):

The classes simply implement IFightBehavior and provide a concrete implementation for fight() in their own unique way. This is their only responsibility.
These three classes are now interchangeable with any return type, property or method argument with the type of IFightBehavior. Like where? Like in Soldier's fightBehavior property!

And finally the implementation and the results when run:

While I'm hard coding the actual setFightBehavior() above, let's imagine that the setting of the weapon came from a weapon select box from a form. When the form is posted you process the selections and dynamically set the behavior and subsequently call soldier.fight().
If you added 50 more weapons to the select box as long as there were 50 more behavioral classes that implemented IFightBehavior, your consuming code DOES NOT CHANGE and you don't have to needlessly subclass and override methods anymore. That's the beauty and power of this pattern - polymorphism at runtime which keeps your API clean. In our example above, calls to solder.fight() need not change.

To drive it home for the last time. A solution without this pattern would most likely force you to add additional subclasses or add if/case's in solder.fight() that checks for weapon a soldier is holding, do some process logic then execute fight() in a specific fashion. Unnecessary complexity.

So there you have it - the strategy pattern in a nutshell. I've managed to work out some pretty slick solutions with this pattern in CF and C# from organizing complex search algorithms to encapsulating gnarly business logic and I'm sure you could find umpteen more uses for it. This pattern keeps complexity low, forces objects to be highly focused with singular responsibility all while being easily scalable and maintainable.

Hopefully I helped you get another step closer to object oriented design pattern nirvana. And remember, as John Whish said to me and I couldn't agree more, "Death to inheritance - long live the strategy pattern!"

-Mick

p.s. - which pattern is next?

ColdFusion Comes of Age - Community to Follow?

2009-12-09T02:05:00.000-08:00

About a month ago my fellow CF'er Mike Chandler showed me this article. The initial once over was quick with only a handful of comments that made me chuckle then I moved on. As I was sifting through old email I found the link and decided to see where the conversation went, if anywhere. That, my friends, is what inspired this blog post.

Basically a CEO announced a launch of their new site which was rebuilt in Java and somewhere in the post called ColdFusion outmoded. I’m writing this entry not because I’m miffed that he called ColdFusion outmoded, I’m writing this because of some, not all, of the responses that ensued. People are entitled to their opinion both the CEO of MFG and the responders alike but I really couldn’t believe the level of back and forth going on between commenters. Some responses reminded me of elementary school where kids are cracking “your momma” jokes back and forth, getting heated over well, nothing. Sarcastic banter and throwing out stats is not going to make your point, especially in a forum. And seriously, all the CEO said was that it was "outmoded". Give me a break, that is not a big deal people.

So this got me thinking, “What do we as a community need to do to be taken seriously?”. Here's my $.02.

First, let’s stop bashing other languages. There's something to learn in every language regardless of what side of the fence you are on and whether you care to admit it or not, its true. There is always something to learn. Always. Secondly, let's stop responding immaturely when someone says “ColdFusion {insert derogatory verb here}”. Let it go. Breathe. Do a 10 count. I get it, we are all passionate about ColdFusion. But we should be even more passionate about what ColdFusion is used for - software engineering. More passionate about solving complex problems that completely kick our ass. The kind of problems that you find yourself solving when you're dozing in and out of consciousness in the middle of the night cause you can't sleep, just to wake up still trying to solve it and can't get that computer on fast enough. And hey, in my experience ColdFusion allows you to solution those types of problems and solution them pretty damn well - thats why I love it just as do you.

Adobe and the ColdFusion team hit the nail on the head with CF9. They’ve armed us with the capabilities to enter the arena and stand and bang like the big boys. It gives us the ability to execute the core tenets of object oriented programming - encapsulation, inheritance and polymorphism. It makes OO feel natural just like C++, Java and C# with its new and improved script syntax and quick object creation. It feels grown up. It feels like it has come of age. It demands to be taken seriously. But there’s one critical piece missing. We as a development community need to come of age with it. We need to mature. We need to start designing before we write code. We need to really understand object orientation and design patterns. Strive to be software engineers, not just complacent code monkeys. We need to advocate designing before we develop because identifying problems with our architecture while it’s in design is much better than discovering them hundreds of hours into development and hundreds of $'s in the hole. We need to design, poke holes, re-design, poke more holes, then lastly start coding (this should be the easy part). But hey even if you need to rework design during development, your design should be such that its easy to maintain and modify. We need to architect applications that scale. That are flexible. We need to understand why to use Framework A, B, and C and more importantly how they work under the hood. We need to fully understand the problems these frameworks were created to solve.

Lastly and most important, we need to solve problems just like the big boys and be sure to share the solutions amongst each other. Problems whose solutions cannot be summed up with “check out how I used this really cool CF feature to create a pdf or make an ajax request”. Other languages and developers don’t care about that. They’ve got their own cool features that they'll argue it puts yours to shame and vice versa. We need to solve problems that transcend languages. Problems that only sound design, architecture, design patterns and well written, efficient algorithms can solve. Once you solved it, share your solution. Get involved. Publish it. It’s only then that we can be taken more seriously. It's only then we can share complex solutions that programmers from other languages can stumble upon and implement. Solutions that came from the mind of a ColdFusion software engineer. A solution that programmers from another language can respect and apply with their language of choice.

CF9 has come of age and has equipped us to do all of the above. It’s been our responsibility as a community to come of age with it. Next time that startup comes knocking, do it right from the beginning. The next time that corporate piece of software needs a new module, do it right from the beginning. Do it object oriented. Use frameworks for the right reasons. Even dive into design patterns. There is a reason for object orientation and blackboxing, that's why its taught in computer science. Don't fight it and know that advice and direction is only a tweet or email away - our community is great about that. Apply those concepts and over time that piece of software you wrote will grow and scale easily and it's then that your piece of software will be a testament to ColdFusion and its capabilities. It's then we'll no longer have to squabble over statistics because we've got proof of high profile adoption.

We're on the path to get there. We have a fricking ton of brilliant minds in this community doing some very cool things every single day, you included.

So let's walk the walk and stop talking the talk. Let's build better software as a community. It'll end up speaking for itself.

-Micky

MXUnit - DataProvider Goodness

2009-12-08T11:38:00.000-08:00

Just when I thought my unit tests were pretty dang light its become even lighter after I came across Marc Esher's blog post about DataProviders that shipped with the new MXUnit 1.0.8 release.

In my unit tests, especially tests that are very data driven like DAO's and validators I found myself doing this alot:

As the screencap shows, there are many time where you need to setup data sets that will be used during assertions or mocks and usually start out inline. You'll find yourself duplicating tests that really assert the same thing just with a different data set. I admit that in unit tests I've become lazy and often times do not even create a private method that returns a pseudo dataset for reuse, yea yea I'm sorry. I just duplicate this code above in each method that needs it because it feels innocuous in a test, I mean after all its simply a query with one record for goodness sake. Well with the new MXUnit data providers its time for me to stop whining because they've made it extremely easy to declare a dataset (DataProvider) and have it injected into the test by doing just three things so:

Create your dataprovider in the setup() as a class level member (variables scope)
Annotating your function using the @mxunit:dataprovider annotation with the value being the dataprovider you want to inject.
Referencing that dataset as an argument in your test method

Here's how it looks like now (with the provider flow):

Booya - that DataProvider is now available to any test method in that TestCase. No need to write duplicate code or private methods. MXUnit simply passes it off politely to your test method. If you have multiple rows in the query it iterates each and fires your unit test, no more separate tests essentially doing the same thing. Damn you MXUnit for making me write better, more efficient code.

MXUnit all the way.

-Mick

Resources

http://blog.mxunit.org/2009/11/data-driven-testing-with-mxunit.html

http://blog.mxunit.org/2009/11/mxunit-108-released.html

Quicksilver Framework - A Primer

2009-12-08T09:13:00.000-08:00

Let's chat about some Quicksilver! Quicksilver was created by a good friend of mine, Brian Carr, who has brought us the first RESTful framework for CF called Powernap. For reasons unknown to me he has given me the honor of working side by side with him on Quicksilver - a framework that allows programmers to quickly create MVC applications with some powerful and flexible features.

When we say powerful features, we mean it. We all know that MVC is powerful and flexible because its given.... No need to stress that. The fact that "MVC" is in the Quicksilver USP is enough. When we use the words powerful and flexible we are truly talking about the features as well. QS offers so much more than just MVC. So... much... more.

At a high level, here are the 4(+1) core tenets that we stuck to when developing QS:

Provide MVC
Native Aspect Oriented Programming(AOP) capabilities
Provide Automatic Dependency Injection
Provide fully functioning REST support
Provide 1-4 easily and transparently as possible meaning there should be no framework artifacts in your actual implementation code getting in your way resulting in an app that becomes extremely dependent on its framework. If we achieved this we can guarantee that your application remains HIGHLY testable which means better and more focused tests, which leads to non-fragile software, which leads to happy customers and ultimately you not having to be a nervous wreck every time a code migration is ordered.

So how did we achieve this? By using Annotations. We took a page from Java/C++ and refer to them as annotations but other popular programming languages have the same feature but named differently e.g. C# calls them Attributes. Although there are many different flavors of annotations from language to language, at the end of the day annotations will provide metadata for your class files, its members and source code. With the release of CF9, you can append this metadata by way of using the cf comments with two asterisks like so:

Notice how the property is annotated via a code comment that this factory property should have a type of "path.to.some.factory". What's so cool about this? It's because an annotation is not part of implementation code - code that goes inside an actual method to be run or as a class member. Anything inside this annotation is available to the developer to introspect at runtime but actual implementation code knows nothing about it. This gives developers the ability at "compile" time to introspect source and perhaps churn out automated documentation, run debug level tests when "compilation" is running in debug mode, or in our case build a framework that gives you some really powerful features like the ones I explained above.

My intent on this entry was not to touch on every single one of those points but lets start with one - Automatic Dependency Injection. If you look at the example above we are simply telling the object that it should have a property called "factory" that has a type of "some.path.to.factory". No dependency injection going on here son, zero. That is unless you've installed Quicksilver and have either Lightwire or Coldspring installed in root. QS will by default look to see if you have Lightwire installed if not it looks for Coldspring. We decided to go with Lightwire first since it manages transient objects better than Coldspring. Ok back to the code - you've installed QS and Lightwire and now when you create your object, QS will automatically inject those dependencies for you. No XML, no bean configuration, nothing. Just the simple fact that you've annotated your property with the type is enough, QS takes care of the rest. So when you ask for an object that has property annotations attached with either the @type annotation or even inline property declaration like - property name="factory" type="some.type", you will have a proper object and its correct dependencies.

But what about for our interface based programmers that love properties that are of an interface type? Properties that doesn't have an actual concrete implementation defined as a type? Good fricking question! Here's how QS supports that with an addition annotation.

Simple type your property to the interface then use the @implementation annotation to give the actual concrete type. Your property still stays polymorphic and the world is good to you again.

So now you see the power of QS by way of annotations. QS exposes all of its features through annotations. We grew tired of having framework artifacts littered throughout code. Controllers that needed to instantiate this and extend that. XML configs spanning hundreds and hundreds of lines. With QS you get MVC without that nonsense and so much more on top of that. And we've tried our best to make sure that the "so much more" is NOT a pain for you to implement when time comes to use them like AOP or REST.

Brian and I are putting QS to the test. You've heard me talk about the new platform that we're writing from scratch. This project uses Quicksilver so know that its being actively developed. I spoke with Brian and we've got an updated QS release coming up soon so please friends do check it out, find some bugs and put it through the ringer. Brian and I are always open to questions and feedback so definitely reach out.

With that primer out of the way, we'll dive head first into the AOP or REST capabilities of Quicksilver. What you think?

Later friends.
-Mick

Using Observer Pattern to Save Composite Objects

2009-12-07T08:30:00.000-08:00

I've recently been asked by a few people how I go about saving composite objects so I figured it may be worth a blog entry. I have a handful of applications I work on, both in CF and .NET, that share similar patterns for solving the problem of saving an object and its composites transactionally and in order. For example, if Object A has an Object B through composition and B cannot save until A is finished saving itself. Additionally if B has a problem saving itself, A should rollback as well. So how the frick does that all work?

As with really anything in programming it can be done a million ways. Some ways are better than others but there is no golden ticket. In my experience, an acceptable solution could be by way of a service object, business object, DAO, ORM (no brainer here) or a combination of all of the above depending on how abstract you want to get. In this example I'll walk you through how I solved this problem using the tried and true Observer Design Pattern.

The Observer Pattern was created by the Gang of Four (GoF) - Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Long story short back in 95' these very smart guys identified 23 recurring software problems, designed solutions for each called a "design pattern" (which borrowed from proper architect Chris Alexander) and released a book to school us on it. If you don't own this book you definitely should, shame on you. Familiarize yourself with the problems and patterns the GoF describe and lock it away in your brain cache. When time comes, you'll be well equipped to tackle some tough design problem thrown at you. Cool? Good. Here is the definition of the observer pattern:

Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically.

So given the composite save problem at hand, I looked to design patterns to see if I could use one as a viable solution. I think I was on a long flight to Boston so I spent the entire flight sifting through design patterns and with the help of the book and a beer or two, I was pretty sure the observer pattern would be a good match. Thinking of the problem at a high level we have a possible one-to-many relationship between an object and its composites that needed behavior between the two such that when an object is saved, it needs to notify the composite objects who are waiting intently to save themselves but cannot before that initial object is saved successfully. For the purposes on this blog entry I've created a couple class diagrams that explain the design, then we'll jump into some code. Here's the high level/abstract:

[click to view full image]

You'll see that there are two interfaces involved in this implementation:

IObservable - Any class that needs to be observable must implement this interface. It contains methods to attach() as a observer, detach() an observer and to notify() its observers of an event. Simply put, if you think of a pub/sub technique, an object implementing IObservable will be the publisher.
IObserver - Any class that can attach itself as an observer to an an object that implements IObservable must implement this interface. It has the API that IObservable will call when it notifies its observers.

Now here is the UML with concrete implementations:

[click image for full view]

Looking at this diagram you should be able to discern the following:

We have a Person that has an Address - here's our composite relationship.
It's a one to many relationship. A Person can have multiple Addresses.
Private member - observers - is of type IObservable[]. Whats the brackets for?
There is a save() method is each object. To keep things simple just imagine that save() is delegating to a DAO or using some snazzy ActiveRecord pattern to actual persist the record. Bottom line is don't worry about it for this example.
Person is observable by way of implementing IObservable. That little symbol with the interface name above each class is called a "lollipop", no joke. It means this class means "implements" whatever name is shown.
Address is an observer by implementing IObserver.
addEmail() has an optional second parameter "attachAsObserver" that is default = true. Remember this, you'll see how this plays into a clean API when using this pattern.

A concrete implementation of Person might look like this:

[click image for full view]

You'll notice that:

attach() will push an observer into the private subscribers array. This method is responsible for allowing observers subscribe to it (Person).
notify() loops over all observers and invokes update("some_event") on the observer that tells the observer, "hey dude, just fired "some_event". In the same breathe it is also inspecting the result of notify() that sniffs for a return of false which means something went wrong so return false and stop notify execution. I know that there are possible scenarios where you would not want to stop processing and move along but exploring that is out of the scope of this example. Although if you go ahead and re-factor this example to do this I'd love to see it!
We've overridden addAddress() which manages our address collection. This is essentially the setter for Address but called it add instead of set because of the one-to-many relationship. Notice that the we've added a second parameter to this method called attachAsSubscriber that has a default = true so its optional. In the logic, we're saying if attasAsObserver is true then automatically attach the Address as an observer to Person.
save() calls super.save(). Make believe Person extends some base persistence object.

K - on to some Address code:

[click image for full view]

Here you'll notice that:

update() will receive the event message and process accordingly. Please do not confuse this with a CRUD operation. As a best practice in you database layer there really shouldn't even be a method called update(), rather call it save(). Anyway, what update() will do is inspect the event that was just fired by IObservable and if it is a "save" event, then return save(). This is a very important piece right here! The return of save() gets bubbled up to the observer. This will ultimately determine the rollback which we'll get to in a second. You can optionally override Address.save() also do the same algorithm as Person.save() if Address also needed to be observable and notify its own observers.But thats up to your requirements. Just want to drive the point home that pattern can be implemented n-levels deep. D observes C who observes B who observes A. A = mack daddy.
update() by default will return true.
update() uses the eventInfo structs observable key to get the Person that was just persisted. This allows Address to save() correctly. I chose to go with a struct for eventInfo because an event might be fired in the future that needed to pass a handful of additional info to its observers. This allows for future flexibility.

And finally here is how you might possibly consume this API in its entirety. For the purposes of this example its in a cfm page but you'd probably want it somewhere in a persistence layer - perhaps a controller/service type object.

[click image for full view]

Here we have:

Created a new Person and Address.
Associated the Address to the Person. Under the hood this will also attach Address as an observer to Person but the API remains simple. Looks just like a simple setter - addAddress(in Address). It has no idea the observer pattern is going on in the background.
We are simply making the entire save process transactional and when an observer fails to save or an exception is thrown, we handle it accordingly by rolling everything back. You could have 100 objects in the queue to be save()'d, fail on save() #99 and it'll nicely rollback everything for you.
And now since we've kept the Observer pattern extremely generic, you can feel free to leverage this implementation for any other event, not just saving.

There you have it. Observer pattern goodness that could hopefully make the GoF a little proud. I've implemented this technique both in .NET and CF and it works great for me. For my C# peeps, Microsoft is releasing an set of Observer Pattern interfaces that will support this type of usage out of the box. Check it out here.

So in closing, this is just one technique of many so I encourage you to explore some other ways and please point me to ways you've done it in the past. I'm interested in seeing it!

Apologies in advance for any bugs in this code. Everything was typed on the fly but hopefully you get the idea.

Till next time my friends.

-Mick

Resources:
Design Patterns with C#
Christopher Alexander, founder of the design pattern in architecture.
GoF, you owe it to them so buy their book

Typed Arrays in ColdFusion...kind of.

2009-12-04T22:10:00.000-08:00

You read it right but I'll warn you now it'll make you feel a little icky.If you are storing a composite relationship via a private array you've probably been using the type of "array" all these years. You can pseudo type your array in CF9 and I even think you could do this all the way back in 6.1 but back then an empty array would throw an error. Anyway,if you choose to do so you can pseudo-type your arrays in this fashion:

Back in 6.1 if that array was empty, it would throw an exception but that doesn't seem to happen anymore. Simple use the type and add the array notation (open-close brackets) and viola it works. I kinda have a type safe array... kind of. It only checks the type of the FIRST element in the array. So I could have an Address object in the first position and an int in the next and it will NOT fail.

Half kudos, half fail.

PEACE.

Mocking - An essential unit testing technique

2009-12-04T11:11:00.000-08:00

Oh the joys of unit testing. Brian C. and I are creating a new platform from the ground up. From the get go unit tests were a standard. You need to write em, if you don't its timeout in the corner for 10 minutes. We started this project about a year ago on CF8 but recently got approval to move over to CF9. Very cool. So we started our script conversion A fricking SAP since the sooner we could rid the codebase of cftags the sooner my wrists and fingers can like me again.

Within that year we wrote unit tests for every single method in our codebase and tried our best to have near 100% code coverage. Life was good, we felt protected. That is, until we started hearing about Mocking during our script conversion. Essentially we were refactoring code to script which made us take a deeper look at the quality of our unit tests - and ehhh it was mediocre. A majority of tests felt clunky, doing too much, finding assertions that were'nt needed in the unit test in question but rather that belonged somewhere else. We both heard about mocking and decided to dedicate a few days trying to understand it and see if it solved our unit test shady-ness. I first heard the term "mocking" by a good friend of mine, Shakti S. who is spearheading the JUnit unit testing effort over at his shop. I'd say within the first few minutes of the explanation my eyes glazed over and my brain was sizzling. During that discussion, one point (out of many of course) that resonated with me was when he said this, "You can see the true power of mocking when writing tests against objects that leverage DAO's". So I slept on it... Shot a few emails back to Shaks and eventually brought it up with Brian as well. Brian and I spent a good day researching mocking (EXCELLENT resource by Martin Fowler- http://martinfowler.com/articles/mocksArentStubs.html) and exchanging conversations that resulted in many "I think I get it. Lets run some tests". About the 10th "I think I get it" later we came to an epiphany. Unit tests need to be as simplistic as possible right? At the end of the day software functions are no different that mathematical functions - you define the inputs therefore you can assert the output.

OK so where does mocking come in? Well the object/method under test (coined System Under Test, SUT) can be delegating to composite objects to get its ultimate result. These composite objects often have their own composition so you can imagine what that setup() of that test case would look like.You'd need to instantiate the SUT, its composite relationships and THEIR composite relationships just to get the SUT in a state where its testable - ugh no thanks. Herein lies the problem that Mocking solves. These composite method calls within a SUT are termed collaborators. You, as the developer who have access to the source code, can mock/fake the results of a collaborator WITHIN that SUT. This is desirable because you should know what a valid expectation can be for that collaboration method. Instead of manually injecting/constructing the proper dependencies every single time just simply mock it up. You should NOT be manually injecting/constructing the proper dependencies because it takes your focus off of the SUT and you end up with a handful of extra code that you don't need, compositions that don't need to happen and assertions that belong somewhere else or worse, doesn't even add value to the unit test.

Back to how this mocking concept clicked as it pertains to the the DAO problem. In our new platform we leveraged an IoC container so we didn't have to manually construct dependencies every test case setup(). However we did have to inject the bean factory into each test case - so same problem, different dress. So how'd this epiphany apply to DAO's? Well when a test for a method that uses a delegate call to a dao method like dao.save(), that actually persisted a test record into the database. This should have just made you cringe... alot. You heard it right. Through each run of the build (firing unit tests) we were persisting test data to the dev database everywhere something.save() was called. This happens through local developer builds as well as continuous integration. We needed a way to make dao.save() return true but don't really go in there and persist records to the database. This is what integration tests are for, not unit tests. So after reaching mocking nirvana the solution was simple - mock the SUT's collaborator for dao, specifically the dao.save() method, to return true. Whatever dao.save() used to do is now overriden. If that save() went on and collaborated with 100 more objects, it no longer has any relevance here. We've mocked it. We've told you, dao.save(), what to return. We don't care how you really use 100 other objects to return true. You, dao.save(), are not the SUT. So I'll make you return true and you'll like it since I'm your daddy and my actual SUT execution path expects you to return true in order for me to assert properly.

So to conclude just remember that whenever you run across a SUT that has collaborators (composition), chances are you'll want to mock it. This keeps your unit tests controlled, light and focused. Keep up with your unit tests as it's tremendously helped us with this tag to script conversion. We'd convert to script, run the unit test where it would sometimes blow up in our face because we wrote some logic incorrectly. Gotta love it.

Hope this helped. I wanted to focus on the abstract and not show code however if you would like to see examples just let me know.

Good resources:

http://www.mockobjects.com/
Martin Fowler- http://martinfowler.com/articles/mocksArentStubs.html
Ask @marcesher on twitter - http://twitter.com/marcesher (MXUnit contributor)