Tuesday, August 11, 2009

The bad side of good abstraction

Good, well-designed abstraction can be bad. Or rather, has some negative aspects as well.

Let me explain.

In most projects I have been involved in some form of abstraction layer was build on top of the libraries and frameworks used. A kind of domain-specific framework.
Typically this layer is meant to make it easier to build the system. The (more experienced/senior) developer(s) would implement this, thereby allowing the rest to work with a limited facade of the lower level libraries. I have also instigated and added to such layers many times; it is especially useful to build a more domain oriented application framework on top of the general Eclipse RCP framework (or Swing, in the earlier days).

If this layer is designed and implemented well, it usually enhances productivity and generally makes everyone feel all happy and content.

After a discussion the other day with a very bright developer, who is still new to Eclipse RCP, I came to realise there is one serious bad issue with this approach: skills development.

This guy has been working on a very complex RCP application, using a lot of the abstracted custom framework and some of the "low level" RCP itself. And then came the day he had to develop an Eclipse RCP app from scratch, in another business domain, without the help of the abstraction layer.

Suddenly he had to start learning a lot of "new" RCP concepts and API. Things that has been hidden (read: encapsulated) away from him until now.

This led me to start thinking about my approach when I mentor or lead teams: I must somehow maintain a balance between this kind of (necessary) abstraction that enhanced productivity, and still allowing the team members to pick up the necessary skills on the "hidden" areas.

Please feel free to contribute your ideas and possible solutions to this balancing game as comments.



6 comments:

Eric Rizzo said...

To mitigate the risk, practice extreme shared ownership of the "domain-specific" layer, so that all team members have experience in developing, enhancing, and bug-fixing in that layer. In my experience, it those kinds of "convenience" layers are bets developed after having used the low-level framework/API for a while and seeing where it can be simplified for your application or domain; make sure to get ALL developers involved as soon as that tipping point is reached so none are left behind.

Eric Rizzo said...

Stupid Blogger doesn't allow editing comments...
That should read "those kinds of convenience layers are BEST developed..."

Herman Lintvelt said...

@Eric, Thanks for the comment. I also find that the convenience layers develop almost naturally over time in a project. The issue I blogged about is more inclined to happen with developers joining the project after this layer has been put in place, in my experience.

Konstantin Komissarchik said...

The situation that you are describing is inevitable. We are not all coding in assembly for a reason. Once someone has built a higher level of abstractions that people can be productive with, only comparatively few people need know the details of the lower levels. That's an important part of efficiency gain in building another layer of abstraction. For any really good abstraction, only a small percentage of developers will know the next level down. How many Java/SWT developers also know C/Win32?

I think the key in realizing most efficiency gain from an abstraction is in making the abstraction as broadly applicable as possible (while maintaining the spirit of the abstraction) and in removing barriers to adoption (make it open source or something). In the example that you presented in your post, I am guess that the developer in question would have been quite happy continuing to use the abstraction that he already knew if he could have utilized it on the next project.

Scott Lewis said...

I believe your notion of 'bad side of good abstraction' is quite close to the notion of 'leaky abstractions'...i.e. Leaky Abstractions. I'm of the opinion that layering of abstractions is one effective way to deal with this problem although admittedly layering of abstractions can result in complexity issues as well.

Herman Lintvelt said...

@Scott. I agree that this is similar to what Joel Spolsky called Leaky Abstractions. I specifically agree with the fact that one has to teach the (new) developers about the layers that are supposed to be "hidden" away by the higher-level abstractions, which all are unfortunately leaky.