Designing with many servers, and one central server
Was thinking about the following theoretical scenario. What might be some good options for structuring it?
- Let's say there are 10 clients on 10 computers in a given physical location that need to share data and communicate, so 1 of the 10 computers becomes a central server for the 10. This server would give the 10 computers a central location by which they could communicate with each other, likely by all accessing the same shared DB.
- Now you have multiple sets of the above configuration, and you'd like all the central servers from above (1 per set of 10 computers) to back up to one totally central backup database on another server somewhere
I guessed that situations similar to the above might occur often in the real world, so perhaps some of you could comment. (e.g. A division within a company would share data with each other on a shared database, but all 100 company divisions would then have to sync up with a single central server.)
Assuming an application with slight complexity, would this require 2 sets of application servers? One acting as middleware on the backup server, and another application server sitting on the 1 computer out of 10 that hosts the database and coordinating those 10 programs?
I ask because it seems like deploying an application server within each 'division' of 10 computers would require a lot of expertise in terms of managing and maintaining their local app server...of course, you could make every single client in the entire enterprise a thin client and have them ALL work from the enterprise's one single central server, but assume that in this case such isn't possible (e.g. the load on that one server would be inhibiting if the load weren't first shared between intermediate app servers; or, the clients wish to operate on their data locally, and only have it sent to the central server for occassional backups and things like that).
I'm sure you see cases like this all the time in your own enterprises...what's a good way to approach thinking about structuring such a scenario, regarding the challenges of sharing the work on a single central server with intermediate application servers? From what you've seen, how do real companies generally approach such issues? (If my questions come across as really convoluted, let me know and I'll try to simplify.)
I look forward to your help on this concept.
Thanks.
Generally, J2EE applications servers are designed to handle theseexact cases. (Clustering,Failover,In Memory sessions replication).
sorry I couldnot figure out what the actual question is?
WOW! Reading my own question a couple of days later, even I'm confused by my convoluted wording...geez...haha...please ignore everything in my previous post
Here it goes again from scratch...
Let's say you have a huge number of thin clients all over the country accessing a central database. You decide that the thin clients would like to have some form of local access to their data instead, because their network to that central database is quite unreliable and slow.
You fear you may have to turn every client into a thick client, which you don't want to do. You notice an important property about these clients...many of them are located on a LAN with a handful of other clients. Learning this, you propose the following:
Rather than having all clients access the central database (where the network is an issue);
And rather than having every single client act as a thick client;
You could keep all clients as thin clients, but their source of data, instead of being a central database, would be a 'semi-central' database located on the LAN. (Because these clients are on the same LAN as the semi-central database, the network wouldn't be an issue as it is with the central database.) At the end of each day (off business hours and where the network can be unreliable), all of these semi-central databases could back up their day's worth of data to the central database.
My question is, assuming that the above conceptual design is set in stone, how would it be implemented?
My guess is that each LAN would have to contain an application server that can coordinate access to the 'semi-central' database among all clients. But it just seems that having to deploy an entire application server on each LAN, and finding the expertise to configure it when that is not available would be overly complex.
Of course, the completely thin client approach would be the simplest, because then there's only one application server that only needs to be configured centrally...once you start shipping out an application server to people who don't know how to use it seems like chaos waiting to happen...
Based on the conceptual design I've described, how would you implement it? I get the feeling that this pattern might be common enough that you would have some deep insight.
Thanks.
Seems to me you are sort of describing one of the peer-to-peer systems that have become very common lately. Napster, Gnutella, BitTorrent, etc. Is that too far away from your expected design?
When you say "thin client", I presume you are speaking of web browsers. Is that correct? If not, what exactly do you mean?
You say that at the end of the day, the 'local' databases would copy their data to the central one. Would there need to be a mechanism for data to be replicated in the other direction? And, are these local databases and the central database this an relational databases (like MySQL, Oracle, MS SQL Server, ad nauseum)?
I don't see a way for you to avoid putting some kind of application server in front of the local databases if you want to have thin clients. The application server could be something as lightweight as Tomcat.
If you are signing up for local database server installation and admin, it seems reasonable to me that throwing the application server care and feeding into the mix isn't too much to ask.
You could also approach the problem as an appliance - you centrally configure the boxes hosting the local DB's and application servers at your central shop and distribute them to the sites. Using remote admin capabilities (via SSH, etc.) you could still own the care and feeding of the appliance centrally.
NASDQ does (or at least did back in 2002) something like this with its systems that connect the market makers to the messaging infrastructure.
Just a thought.
> Seems to me you are sort of describing one of the peer-to-peer systems
> that have become very common lately. Napster, Gnutella, BitTorrent,
> etc. Is that too far away from your expected design?
I don't see how P2P facilities would immediately address his scenario given his desire for 'thin clients' and my assumption (poor as it is) that that means web browsers.
> > Seems to me you are sort of describing one of the
> peer-to-peer systems
> > that have become very common lately. Napster,
> Gnutella, BitTorrent,
> > etc. Is that too far away from your expected
> design?
>
> I don't see how P2P facilities would immediately
> address his scenario given his desire for 'thin
> clients' and my assumption (poor as it is) that that
> means web browsers.
This is true. I had forgotten that "thin clients" does tend to mean browser-based clients. But it still smells like Gnutella to me.
Gnutella? Isn't that the yummy smelling (and tasting) hazelnut and chocolate spread? ;->
What I'm describing is more of a business application. (By thin client you're correct that I mean browser. Also, they are one way updates to the central database, and all are relational DBs.)
Imagine I hypothetically develop a wonderful little program for businesses to manage their affairs, and I'd like to make it a browser-based thin-client application.
Of course, end users that use the browser application wouldn't need to install anything and can access all their data from the central database on my central servers. If there's a problem with their network however, they can't acccess their data for that time. Hence, I propose the idea of the semi-central database where there's one local database that can be shared among all users in that particular business...that would then be backed up to the central database as needed.
My question is whether there's any way to bundle everything into a simple installable program for all end users so that they don't have to install or configure anything? For the end users, the program would (instead of being a browser, be a simple interface to their local database), and for the computer that has the database and application server, it could be bundled into a program that bundles the database and application server with its version of the program, and keeps that business from ever having to worry about configuring anything...they just install it and their 'thin' clients can start using it right away...
Is such an implementation possible? What things could I look into that I could learn more about how to bundle and integrate everything like this so it keeps the customer from having to worry about the details...
What other interesting ways of structuring something like this might you suggest?
Thanks.
P.S. Interesting idea re: bundling it into a box, and while I don't think it's feasible for this particular need, I'll certainly look into it.
> Gnutella? Isn't that the yummy smelling (and> tasting) hazelnut and chocolate spread? ;-><Homer>Mmmm... Gnutella...</Homer>
A very interesting topic.
For deployment you can take a look at Java Web Start. It's a single click deployment solution for Java desktop applications. Basically you just bundle your application into a jar file and place it in a web server. After that it's just a matter of configuring your JLNP file to point in the right direction.
I must admit that I've never thought about having multiple databases before. I'm curious about one thing though. What about your business logic? Since you're going for thin clients, where would you place the code for your logic? In multiple application servers connected to each secondary database or you're going for stored procedures in the databases?
I've looking into a similar issue in my own company. We have 1 database server, 1 application server and over 7000 browser based users with heavy transactions. Let's just say the performance is less than satisfactory. (>^^)>
The clients would still be thin, but the computer that hosts the intermediate database and application server would contain the business logic. I haven't actually thought about the stored procedure option yet because I don't know enough about it -- but any stored procedures acting as business logic would be in the secondary database (the central database would act as a backup mechanism).
Regarding your particular situation, I'd like to ask a couple of questions to you about it because it's so similar to what I had in mind when asking this question:
What's the nature/type of your application? If it's important for your 7000 customers to have access to it to run their businesses, wouldn't you be in deep trouble if your network goes down (because they can't access any of their data), or even if their ISP or router goes down because they can't access any data? How do you deal with this on the technological side of things? What about the business side of things (how do you incorporate these risks into your SLA for your customers)?
My biggest fear about a pure thin client scenario such as yours is that what happens if the network problem is on your side of things? You'd have 7000 ANGRY customers because none of them would be able to get access to their data. This would be 7000 times as bad as if the ISP for one of them went down. I'm sure this must have happened at least once in the history of your company...could you describe how you dealt with it, and the reactions of your customers? Wouldn't this have been enough to make any pure-thin client application provider reconsider their architecture?
Do you offer some kind of localized data option where they can operate on their data locally and offline when the network is down?
Please do let me know what strategy you come up with in the end to tackle the issue.
Your assumption is correct. Our customers depend greatly on our system. They run their businesses by executing all their transactions through our system so in the event that either our system or the network goes down, it means no business. Yes, we've encountered this and yes, there was hell to pay.
We're still debating on a solution but the general direction right now seems to be moving back towards a thick client architecture. We'll deploy individual desktop applications to our customers together with the business logic and some of the data in XML/Text files. The idea is to minimize the server round trips made by each customer for every transaction and theoretically improve the performance of the system. Since both the logic and the data is installed locally, the system would still be able to function in the event that the network goes down. (at least for a while)
So in comparison with your idea, instead of having several secondary databases we'll have over 7000 local data storage clients. We'll synchronize the data to the main DB at a non peak hour in batches.
I don't really think we're solving this in the right way. We're not really up to date on the technological side so there may be something out there that provides a better solution as well. Anyway, what's your point of view? Oh and everyone is welcomed to join in as well. (>^^)>
