Big Picture Architecture - why we never go far enough
I抎 like to start some discussion about some bigger picture architectural concepts.
I guess what I want to talk about stems into the approach of software development in general. I have a particular philosophy for design of systems: that most code can be simple, generic and reusable. It抯 the basic precept of object-orientation, but how often do you see developers happy to use high level classes, but then "hard-code" just about everything else. The argument is that things can not be totally genericized, because there are always exceptions to simple rules. My approach is that most things can be automatic and still allow forks to be implemented where needed. Most people I suggest this to think I抦 crazy (perhaps I am, but I thought it was for different reasons).
My project Atomic Objects aims to create an object environment which creates a virtual object space in which dynamic interconnection of objects can occur. It抯 basically an inversion of control environment, on steroids.
http://atomic.catchpole.net/
I admit that a fully dynamic environment is an all or nothing proposition. It must provide enough features to be useful or it becomes more a hassle than it抯 worth. Just like WYSIWYG editors. A nice idea, but useless if they trivialize the situation and don抰 supply the fine control you need for real life applications.
I guess you could call it "moving patterns into the framework", or Model Driven Architecture without all that messy code generation.
These concepts may be "out there", but it disturbs me how most developers refuse to think out of the box.
Christian Catchpole
_
Atomic Objects
http://atomic.catchpole.net/
Sounds more like an IoC framework on crack. You're no Rod Johnson.Looks like a blatant, lame effort at self-promotion. Do your advertising elsewhere, eejit.%
It was self promotion, but about something I beleive in. And it's open source.. Isn't this what forums are for?
If I was Rod Johnson would I be permitted to discuss this topic?
> It was self promotion, but about something I beleive in.
IMHO, that's what the problem is: you believe in it while it lacks any
scientific foundation. Your site immediately reminded me of another
'thing' that lacks any scientific foundation. Have a look [url=http://www.beautifulcode.nl]here[/url].
JosAHa at 2007-7-15 23:06:25 >
Fair enough - I agree my site is a little thin on the ground. And all that organic binding stuff on beautifulcode looks swish, but means nothing in practice.
I can understand that my site reminds you of a 搕hing?that lacks scientific foundation. There isn抰 much on there to say otherwise. But I don抰 agree the idea has no basis. All I am suggesting is taking IoC and adding persistence, network replication and patterns within the control layer. So that all objects can be handled in a consistent way.
I抦 not saying I have all the answers, but my early prototypes are working well. I can create an environment in which objects can be created and bound via compatible method calls?And then persisted and replicated and all that good stuff. Will it be good enough to solve real problems? Only time will tell.
I might just stay out of the forums until it抯 performing a few more tricks. I didn抰 expect to be called a 搇ame idiot?in an architectural forum.
> Fair enough - I agree my site is a little thin on the ground.
A little? 8)
> And all that organic binding stuff on
> beautifulcode looks swish, but means nothing in
> practice.
I think we called a lot of these ideas Jini back in 1999 when Bill Joy was talking about self-discovering objects.
> I can understand that my site reminds you of a
> 搕hing?that lacks scientific foundation. There
> isn抰 much on there to say otherwise. But I don抰
> agree the idea has no basis. All I am suggesting is
> taking IoC and adding persistence, network
> replication and patterns within the control layer.
> So that all objects can be handled in a consistent
> t way.
So what? Sounds like buzzword heaven to me.
>
> I抦 not saying I have all the answers, but my early
> prototypes are working well. I can create an
> environment in which objects can be created and bound
> via compatible method calls?And then persisted and
> replicated and all that good stuff. Will it be good
> enough to solve real problems? Only time will tell.
Indeed. The marketplace will judge you and your work.
> I might just stay out of the forums until it抯
> performing a few more tricks. I didn抰 expect to be
> called a 搇ame idiot?in an architectural forum.
The difference between you and Rod Johnson is that his Spring was the outgrowth of the framework he built up during his years of consulting. It actually worked in practice before he brought it out for others to use.
You've reversed it: you're a solution looking for a problem. How many other folks are active on your SourceForge project? How many clients are running this in production? Are you invoicing with this? That's a big difference, IMO.
%
And who will take a software site seriously when it has links to Bianca Beauchamp, their "test model"? Please.%
"I might just stay out of the forums until it抯 performing a few more tricks. I didn抰 expect to be called a 搇ame idiot?in an architectural forum."
Where to start?
The subject line, for one: "why we never go far enough". Who's "we"? Who decides what is "far enough"? What does "far enough" look like - your framework?
The lame part is using this forum for self-promotion. I think the agreement you sign when you get your ID prohibits such things.
There's little or nothing new in all those buzzwords you trotted out. Sounds great, but those ideas aren't yours.
Every new idea starts outside the mainstream. They work their way in over time as they demonstrate success and acquire mindshare. You give no indication that this passing shower has convinced anybody but you.
The "best" ideas don't always win, either. I'd say Jini was brilliant work, but it hasn't acquired much mindshare. I'm sure I can find projects out on jini.org, but I don't think it's as mainstream as a WebLogic enterprise app server.
Keep working at it. Maybe someday I'll be buying your books. But not today. I'd rather invest my efforts in mastering Spring.
%
I must say I didn't expect these responses when I wrote the original message - but it's all good. I am taking notice. (even if it hurts my little heart and makes me feel 2 inches tall).
I had been working on the project quite a while before creating the site or even releasing some source code. I didn't want to go out there too early, but it seems I have still done it prematurely.
I don抰 think I claimed they were my ideas, or that it hadn抰 been done. In fact the FAQ sais.
Hasn't this sort of thing been done before?
Yes and No. There is nothing new about managed object environments. What is important is how they are implemented. If the fundamental concept of how the system works is too simple, the overall benefit is trivial. If the concept is too complicated, the system stalls under the weight of it's own complexity and need for human intervention, configuration and new code to make it all work. Atomic aims to find the right balance. Simple and elegant enough that interacting with objects becomes about "what should be possible". Yet, not so simple that the end result of these interactions is trivial.
卆nd I havn抰 claimed to have that answer yet :)
There are some sub-modules within the API that are fully functional, and unlike anything I have seen.. I'm not saying equivalents don抰 exist, I just cant find them. See below..
The rules say commercial links.. but I agree in the spirit of the meaning, my links may as well be... I'm trying to sell myself :)
I think I am going to take all your suggestions by cutting the sales pitches for now (and maybe the kinky pictures) and delivering more practical (and working) examples.
-
- Dominion maps object bean trees to a DOM Document using on-demand reflection. Dominion wraps objects from the default DocumentBuilder to create nodes which automatically reflect sub-objects within an object tree, on demand. This allows transformers to selectivly traverse the object model, without the overhead of reflecting the entire tree.
Class Erasure - Creates dynamic classes based on pre-compiled prototypes allowing erased (modified) types and methods for reflection free proxies. And based on this, Bean Fusion, creating method proxies without reflection or CGLIB.
The telnet console and a whole bunch of other stuff.
> I抎 like to start some discussion about some bigger
> picture architectural concepts.
>
> I guess what I want to talk about stems into the
> approach of software development in general. I have a
> particular philosophy for design of systems: that
> most code can be simple, generic and reusable. It抯
> the basic precept of object-orientation, but how
> often do you see developers happy to use high level
> classes, but then "hard-code" just about everything
> else. The argument is that things can not be totally
> genericized, because there are always exceptions to
> simple rules. My approach is that most things can be
> automatic and still allow forks to be implemented
> where needed. Most people I suggest this to think I抦
> crazy (perhaps I am, but I thought it was for
> different reasons).
Any thing can be generalized. The problem is not that it can not be nor that there are exceptions.
The problem occurs when generalization occurs when there are no real examples of how it might be used in more than one way.
I worked on a data layer that the original author had thoughtfully added blob support in in such a way that it could be used conviently in the future.
Only one problem though. There was any need for blobs at the time when it was first created. And after several years there was still no need for it. And absolutely no one could up with a scenario where it would ever be needed in the future.
So the original auther created something that was not used and, as history demonstrated, would not be used.
Further is was also clear that the author had not tested the code. It simply would not work the way it was written.
Do you work for a company that has the following attributes and will you continue to only work with a companies with the following attributes?
- Everything is unit tested.
- All unit tests are maintained and run as part of the check in process
- Requirements/features for future versions are known for at least 2 major and several minor versionsin the future. (With a time line of 18 months.)
- All code is reviewed.
Given the above then I would say that you should be generalizing your code. You know where to focus and can accurately predice future usage.
But given, at least for where I work and interview, more than 99% of companies do not do any of the above.
And further given that I do not have any supernatural powers that let me see the future it would be entirely inappropriate for me to over-engineer my code is the vain hope that it might be useful sometime for some one. Because I know that due to the lack above the most likely outcome is that such guesses will, in the vast majority of cases, lead to code that is never used, and which is more complex and thus more costly to maintain. And that excess cost for the excess failures to predict will more than offset any small advantages that any of my few successful guesses will provide.
> The problem occurs when generalization occurs when there are no real examples of how it might be used in more than one way.
yes, I agree. I have example objects called Cat Factories and Scarf Factories. Cat Factories make Cats and Scarf Factories make Scarfs from Cats. Nice and simple but the problem here is that no one creates Cat objects in reality. Using realistic examples helps guide the real functionality. And a few disparate examples help too, because it proves the one model can work with totally different business cases.
> Do you work for a company that has the following attributes and will you continue to only work with a companies with the following attributes?
Yes, I have been developing 揵ig important?systems for 10 years. Transaction processing and security systems. For Webpay / eFunds and now Australian Emergency Services.
> Given the above then I would say that you should be generalizing your code. You know where to focus and can accurately predice future usage.
That is the core of it. If it抯 not generic, it won抰 work. But it has to be forkable as well so tricky things can be done. Anyway, I抣l stop with the buzzwords.
> > The problem occurs when generalization occurs when
> there are no real examples of how it might be used in
> more than one way.
>
> yes, I agree. I have example objects called Cat
> Factories and Scarf Factories. Cat Factories make
> Cats and Scarf Factories make Scarfs from Cats. Nice
> and simple but the problem here is that no one
> creates Cat objects in reality. Using realistic
> examples helps guide the real functionality. And a
> few disparate examples help too, because it proves
> the one model can work with totally different
> business cases.
>
> > Do you work for a company that has the following
> attributes and will you continue to only work with a
> companies with the following attributes?
>
> Yes, I have been developing 揵ig important?systems
> for 10 years. Transaction processing and security
> systems. For Webpay / eFunds and now Australian
> Emergency Services.
None of what you mentioned addresses the specific attributes that I listed. Big companies and projects have the same problems as small ones.
>
> > Given the above then I would say that you should be
> generalizing your code. You know where to focus and
> can accurately predice future usage.
>
> That is the core of it. If it抯 not generic, it
> won抰 work. But it has to be forkable as well so
> tricky things can be done. Anyway, I抣l stop with
> the buzzwords.
And what does that have to do with the required attributes that I listed?
I work agile / extreme. I'm not sure what you want me to address. My code has JUnit coverage and uses lots of Interfaces. And pretty much every component can be instantiated and run or tested independently. I always thought the mark of a good system (whatever it is) was that you could run two entire systems within the one JVM. No badly implemented singletons subverting the OOness.
"Frameworks" can create problems in testing what is run within them - hence the need to JunitEE with J2EE.
> I work agile / extreme. I'm not sure what you want
> me to address. My code has JUnit coverage and uses
> lots of Interfaces. And pretty much every component
> can be instantiated and run or tested independently.
> I always thought the mark of a good system (whatever
> r it is) was that you could run two entire systems
> within the one JVM. No badly implemented singletons
> subverting the OOness.
All of that is really cool. Nothing to do with my points though.
Do you intend to refuse any job for the rest of your life which does not have that methodology?
As I already pointed out most of the companies that I have seen (not just worked at) do NOT do that.
So if you are taking some moral stance for now and in the future that you will not work for a company that doesn't follow that then your style works.
However if you end up working for a company that doesn't do that, or if someone else is convinced by your argument, despite not working for a company that does the things I listed, then that too is a problem.
Maybe I'm a bit off here, but this reminds me of what is a paradox in software development. The more specialized and adapted a system is to current business processes, the more value it adds. The more generic and dynamic a system is, the more it adapts to changes over time. It is genuinely difficult to write something that is both.
As an example, I can hire 10 geniuses and run the entire Accounting department in Excel. This 'architecture' will be very resilient and accomodate change easily. On the other hand, I can hire 10 software programmers to automate current tasks and 'hard-code' existing business rules into a custom system. Now 1 unskilled person (for example) can do the same task. However, I will have to re-hire my 10 developers for changes to be effected.
- Saish
Saisha at 2007-7-20 17:58:07 >
> Do you intend to refuse any job for the rest of your life which does not have that methodology?
I guess so. For the same reasons I currently refuse non-Java work and dead end projects - why go backwards?
Your points are all cool. I have been only working for such businesses.
I've been lucky in that I have had the freedom to set such standards, so we always worked under the guidelines you mention. I have been working for government recently which has its challenges, but luckily I have the freedom to implement agile methodologies in our team.
Anyway, I thought I would let everyone know that I have taken my project web site offline. All the source code will be available soon on my main site, but none of it will be presented as a "product". I am trying to be a little more humble and prove the aspects of my ideas through example, not sales pitch. And if people want to use only parts of the API that is fine, rather than trying to create a "monolyth" product which just isn't there yet.
Thanks for everyone's verbal spankings. I have been put in my place. It's all good :) I shall now take my bat and ball and go home... :)
> It is genuinely difficult to write something that is both. I guess this is the challenge - to create a methodology which allows for both. by allowing you to fork in the customizations more easily.. anyway, I don't claim to have all the answers.
Christian,
Hmmmmm, where to start ?
In the first post you wrote
<<
I have a particular philosophy for design of systems: that most code can be simple, generic and reusable.
>>
That does not sound like a philsophy for design. Although I recognize the difficulty in saying what design really is, it is not code.
I am not writing to criticize, but to observe that a bunch of classes and methods do not a philosophy make.
Rich Morrison
> I must say I didn't expect these responses when I
> wrote the original message - but it's all good. I am
> taking notice. (even if it hurts my little heart and
> makes me feel 2 inches tall).
I have no clue about the technical concepts you guys are discussing but I cannot countenance the vitriol with which the original poster was met. When I read the first post in this thread I was not even remotely offended and I cannot imagine how it could be that bad.
I could not open the link to the guy's site for some reason, so I do not know to what extent this was a commercially motivated post, but it does not feel that commercial and really I doubt that a flagrantly (but merely) commercial post would have evoked such a response.
What I am actually looking at here is an ideological war, right?
Also I sort of resent the implicit statement that if you can't sell it, it is not only of no commercial value, but totally worthless. The work I do for a living has nothing to do with programming and I code Java just because I get a kick out of it. I suppose that because I will never write code that makes someone a million bucks or become a famous name in the software engineering world I should stop?
Drake
> And further given that I do not have any supernatural
> powers that let me see the future it would be
> entirely inappropriate for me to over-engineer my
> code is the vain hope that it might be useful
> sometime for some one. Because I know that due to
> the lack above the most likely outcome is that such
> guesses will, in the vast majority of cases, lead to
> code that is never used, and which is more complex
> and thus more costly to maintain. And that excess
> cost for the excess failures to predict will more
> than offset any small advantages that any of my few
> successful guesses will provide.
I struggle with that everyday. I have to reel myself in because I constantly want to add functionality that isn't really necessary and may never be used. A lot of the time the desire for that functionality results in overly complex and convoluted code that may interfere with the functionality that was actually important.
For better or worse I'm finding that the key is simply to code the functionality you need and do your best to allow any new functionality to be extended later. I guess it's more about not painting yourself into a corner than it is providing for all possibilities. As long as the code doesn't impose an unnecessary limitation then you can always add it later.
