The Learning Journey

This has nothing to do with comparing 8 different frameworks.

It’s just to tell the horror story of one of our down-the-drain projects.

The client came with an old product that he needed to add new features to. The product does, what is best described as CRUD operations. Nothing challenging, nothing difficult. It maintains a set of data for various client in a certain business domain. Their business model (at least what they’re trying to do) is through ad and subscription revenue. They were trying to move the business forward through enhancing the provided service.

The only challenge can best be described as that the product was a pure mess.

It has been built using (last time I counted) 7 different Java frameworks, some redundant, some obsolete. Apparently it has been the greatest learning experience for few individuals. The product took 8 years in the making, 4 different teams (non remained with the project), 14,000+ files, absurdly normalized 100+ tables database, absolutely 0 documents, No unit/function/otherwise tests to reach its current state.

To give some example of code horror that we had to go through, some Java classes contained business logic, HTML code, display logic, data access mechanisms, hard coded urls. The objects were then instantiated from within JSP pages (with lots of other embedded Java business logic/code) and passed parameters and servlet controls.

This is not to mention the build scripts that took 3 hours to update code off of the CVS, then threw numerous errors during the “real’ build.

That’s in addition to the project tree which was in a state of absolute mess (40+ different individual modules, with some exact modules placed in different locations).

And have I mentioned the “wrapper” PHP application that was delivering a CMS-like functionality (another set of mess, with no cross-application awareness of any kind).

Again, the product didn’t do anything challenging. Just basic CRUD and a couple background processes.

Our initial attempt was, of course, to build on top of the insanely absurd code-base, but after few days, we’ve decided (us, client, operations) that it might worth to venture into a new platform, in order to make it easier (and cheaper) to deliver new business service and at the same time, incrementally migrate features into a more robust code base.

Few months into the making; including the client formulating his requirement and reverse engineering the legacy system, we had our first release of the new business features (including the integration with the original application, modelling 70% of the legacy schema, a streamlined version of few original functions, documenting finds of the original application).

Few more months of client acceptance testing (and repetitive requests from our part for a feedback), client vacations, and [seemingly] client running out of budget, the client finally came back to us deciding that our solution was not accepted.

I frankly didn’t know whether to laugh or cry.

Despite the fact that the client admits, in his own words, the original Java application is a bag of “s$%t”, he refuses any attempt from our side to tell him that his application is non-maintainable.

He decided (and probably promised by some other individuals) to monkey-patch the code base that he has. He, of course, threw our choice of Ruby into the bag of reasons not to pursue this further.

Although I can defend our decision to resort to an incremental migration, and taking Rails as our vehicle quite well, yet, I now believe that the choice of technology should be the least of his worries. The business decision of incremental migrations (or even rather a rewrite) remains the optimum choice in my opinion, whether it’s done in Ruby, Java, PHP or brain*^%&.

The moral of the story is: As long as you care, never be a door mat, insist on getting your ideas through, always believe that you’ve been hired not for your typing speed, but for your intellectual abilities and what they can bring to the business.

Working on maintaining and modifying a five year-old Java application, I came to realize there’s a strong relation between legacy and pyramids. Here’s an analogy:

Pyramids are HUGE monuments, built for the sole purpose of being a tomb to one person. Although the project tree is HUGE (14000+ files), the purpose of the application is amazingly modest (CRUD + search)

Pyramids required a lots of resources to build (time, wealth, labor). The application took 5 years, 4 teams and lots of $ to build.

No one knows exactly how the pyramids were built. Compiling/Building the application is a black box. “Just invoke those commands - twice - and you’ll get your ‘war’ files”, said the Build Master.

Ancient scripts are quite scarce inside the Pyramids. Project documentations are almost non-existent

Sometimes you wonder what motivated people to build such a thing, in such a shape. The schema is highly normalized (talk about contact information spanning across 4 different tables, with email field divided into name and fqdn) - WHY?!

When you look closely, Pyramids are built by stacking layers of stone blocks on top of each other. The application is a grand collection of all sorts of frameworks (counted 4 so far), libraries (uncountable), tools, architectures.

Currently, some Pyramids aren’t structurally sound, and visits to them are quite limited in fear of falling rocks. Dare adding new code to that monster?

Imperfect pyramids can be found along side perfect ones, proving that it took builders decades to master the way they were built. The application contain all sort of “imperfect” code as a proof of how the builders gained experience over time (JSP files with Embedded Classes, with embedded Java code and hard coded urls, business logic, view logic and data access mechanisms)

People visit the Pyramids for few hours as a source of great admiration and respect. Then, continue on with their lives. The application’s code is a source of respect and admiration, but when it comes to something functional, better build something else.

Pyramids can’t be maintained. The great conclusion.

UPDATE: Some people have been reported to complain about a sense of suffocation or claustrophobic symptoms when exploring the pyramid internal passageways. The same is experienced while reading the application code.

I’m trying to handle an integration between a humongous code-base Java application (doing basic stuff) with a new Rails application (talking about injecting Rails transparently). Handling login (multi-application logins) was my first obstacle, and after talking with Steven and Brenton, I came up with the following implementation:

require 'rubygems'
require 'hpricot'
require 'mechanize'

class WelcomeController < ApplicationController
...

  def login
    agent = WWW::Mechanize.new
    page = agent.get('http://example.com')
    login_form = page.form('loginForm')
    login_form.username = params['username']
    login_form.password = params['password']
    page = agent.submit(login_form, login_form.buttons.first)
    cookie = agent.cookies.first
    cookies["JSESSIONID"] = {:value => pet_cookie.value, :path => ‘/’, :domain => ‘example.com’}
  end

…

end

Of course this is just the basic stuff (no error handling, no response checking, no cookie review, …)

The idea is to use Mechanize to simulate the browser actions from within the controller, log-in to the second application, set up a first-application session cookie with the same name/value pair as that of the returned cookie from the first application. I’m not sure that this covers all the basis but it’s not a bad start. Muhammad has also mentioned the idea of having Rails session cookies written with the same name/value pair as those of the Java application.

Thoughts?

Last Friday was my Ruby on Rails presentation at Commerx Corporation (in Lunch and Learn series). It wasn’t as good as I wanted, but, wasn’t too bad either. Some keys on the laptop were in the on-again-off-again mode, so, I had to rely on an external keyboard and use the touchpad at the same time (not a very recommended setup).

Anyways, I think some of the guys were entrigued enough to, at least, give it a try. Being mainly at a PHP shop, I’m still not qualified to do a full comparison, but, from the little time I spent with PHP (so far), I’m becoming even more attached to the beloved Rails.

I think I’m becoming more of a Rails fanatic - comparing every code and architecture I come across to Rails. I find myself asking, how ‘that’ could be done in Rails. How could ruby shine in such a situation. How elegant would it be to implement that in Rails. How much less lines of code I could reach that same setup.

My next project is a minor refactoring of what seems like a complex Java project (read: reusable components). I’m still exploring the source tree. I can’t help but to wonder about the size of the project, and what it ‘actually’ do :). I’m having a hard time forcing myself ‘NOT’ to think of it as a Rails rewrite opportunity. Hope I can manage

Breaking the silence - this was posted by “tprentis” on IBM forum for DB2 on Rails Toolkit

I'm a recent convert to Ruby on Rails. My organization has several (over 50) iSeries machines on our WAN. As MIS manager, I plan to start pushing RoR development in the near future but really need that DB2 iSeries connector running. Please escalate the development effort. Thanks!

So, provided the info is real, and skipping the part that a connector/adapter is not ready up until now, an organization with over 50 iSeries machines (most probably legacy) will be pushing RoR as a part of its technology stack. That's big.

A reply from “Milhouse”

It's not that hard to hack up an adapter if you don't mind using the DBI:ADO adapter (see my earlier post). I've used it experimentally with success. Another option I plan to explore is to use JRuby with their JDBC-ActiveRecord with the existing iSeries JDBC drivers. This has the added advantage of easy integration with Java libraries.

I think it's time to give JRuby a try.

powered by performancing firefox

I have attended the First EGJUG meeting and it was a good kickstart. EGJUG is an attempt to bring together Egyptian Java enthusiasts, help promote Java in Egypt, and raise the skills of Java developers.

With 290+ members and only around 10 attending the first meeting, the group needs a hell lot of marketing. There's also a good chance the Java developers around aren't that interested in such groups/meeting/events. I experienced the same lack of interest when trying to establish a meeting among Alexandria Java User group. That said, I think EGJUG have a better chance given the current mindset and experience of moderators, and core team.

There's an initial plan to invite a different speaker each month, so, I'll be trying to reach out for some experts for Java-related talks.

If you have an interest in either talking or attending, or have any other suggestions in that respect, I'm all ears and eyes.

According to Marcel’s request in a recent comment - here’s how I hacked the list of recent entries and comments on my blog.
It is based on someone else’s work (I forgot who).

Create two new Recent Entries and Comments template pages.

Recent Entries template:


<ul>

Recent Comments template:


#if($website.allowComments)

Include these pages in their respective locations in the main weblog template through #includePage(”<_templateName>”)

It’s just a quick hack, there are better ways to do it (macros), and the pervious method will limit the number of entries according to the page you’re on (try clicking any of the old entries and check the new list) - but these will work for most of the cases. Also, there’s this showRecentEntries macro definition that can be used - but I haven’t tried it. HTH.

Yesterday I had a phone interview slash discussion with the CEO of href="http://www.globe-soft.net" target="_blank">Globe Soft (I
later knew that he's the CEO from their website, he didn't introduce
himself). To cut the story short, the advertised position was for a
senior J2EE Developer, a few minutes into the discussion I recognized
that the company wanted a code monkey rather than a “senior” staff ( href="http://sethgodin.typepad.com/seths_blog/2005/11/great_marketers.html"
target="_blank">“senior” is indeed a generic word). As I was
basically too expensive for the position (around 150% of their pay
range) he followed a negotiation strategy along the lines of:

• Industry average [for a code monkey] is around half of what you're getting.
• You'd gain a very good experience working on that project.
• You have only X years and these are valued at Y value (proportional)
• You currently work at a manufacturing company.

<shameless plug>Despite the fact that he was trying to buy low with some unconvincing arguments, and that I didn't sell myself too hard by neither stressing the couple
of Master's degrees + 8 years sound experience working for both large and small organizations, nor mentioning the consultancy spot jobs, knowledge, passion, etc…</shameless plug> I should stop at one of his points:

“Working for a manufacturing company”.

I've heard the argument before, and I have to agree that working in an industrial/manufacturing company could sound like an “ordinary” life
for a developer. IMO this should not carry the impression that such an environment results in an experience-less time (for the lack of
a better wording). As a matter of fact, one of the key drivers to my love for my profession is that I experience, first hand, how the IT (and more
specifically development) could shape up a business and how they could transfer it from a state to a better one. I also, on a daily basis, HAVE TO deal with the good, bad, failed, unrecognized implementations.

With this in mind, It's impossible to deny the gains:

• Being immersed in user experiences
• Understanding and valuing the meaning of agility
• Gaining domain knowledge
• Dealing with a plethora of personalities with various level of expertise, cultures.
• Always finding Improvements for current solutions (as the interface with
  customer/client never ends).

Unfortunately, I only came across a few (in fact one) who shared a similar experience during his career.
So, If you have worked/been working in an Industrial/Manufacturing environment, I'd really appreciate hearing from you.

As always, a wonderful learning experience and a true gain.

Thanks Mr. Galal.

(I will post this under java category for maximum readership. Apologies).

UPDATE: I've received many good and positive responses to my post on the favourite “The Business of Software” disucssion group.

JRoller haven't reset the hit counter on my blog for a couple of days (or those of other blogs too as I can see). Is this a feature or a bug? Should I be resetting the counter explicitly?

I'm starting to wonder what more can Freemarker offer than JSP 2.0 (or even 1.2) with JSTL EL (Taglibs)?
Still it isn't clear separation (as claimed by FM folks) between the Ms and Vs.

Moreover (for someone like me), embedding JSP Tag libs (being it Struts, JSTL, or the like) into Freemarker templates is equivalent to using JSPs with JSTL but with the subtracted benifit of adding java snippets to the page.

OK, OK, I can hear screams about the last sentence, but, what's the harm in <%= (new java.util.Date()).toLocaleString() %> inside a JSP? (I can't still figure out how to echo the current date in Freemarker)

One more thing - Does the word “Specifications” mean anything?

Would like to hear opinions about Freemarker.