Configurable software choices
Background
Many of us think that they would be better off not being tied to a technology.
We hear statements like these :
“Just make software choices configurable”.
“We do not want to be tied down to any technology”
What ? — A Simple SQL database Example:
So here we have a application service(microservice?).
We would love to have a choice at install time of using Oracle or Postgres.
We would hear statements like
“After all both are ACID databases”
“How hard can it be ? SQL is a standard”
Aspects to consider
We will discuss some of the aspects, we will face when dealing with an application working on configurable software choices.
Technology 1 —JAVA
Background
We will start with a lightweight read on java to drive home the point.
Java helped popularise the phrase “Write once, Run anywhere”. It was revolutionary at the time.
Our app would just work the same way anywhere.
A) Oracle Java 10 is available for the following platforms :
Linux , MacOS, Solaris SPARC and Windows
B) IBM java 8 comes for the following platforms :
AIX,Linux, z/OS and IBM i
Hence we can assume Java runs the same on all these platforms ? Maybe …
We do know, there can be aspects of difference between the two big java vendors, IBM and Oracle:
Oracle , IBM two biggest java implementers, do have differences from time to time
IBM and Oracle are not the only JVM implementers, have a look below for so many other implementers :
It would be incorrect, to assume there is no difference between the JVM implementations
Conclusion
Java does run on many platforms, however it can have nuances depending on the platform it is run on. We need to be aware of this reality.
My view we should run in immutable containers like docker.
Technology 2 — Databases
Background
There are a raft of SQL/NoSQL/NewSQL Databases.
Configurable Datastore
This is a requirement, usually when we are in the business of selling software to multiple clients.
Say one of our potential client is an Oracle shop, another client is a DB2 shop and yet another is a MSSQL shop.
If we can say do not worry our product works with all of those databases and also works with NoSQL databases like MongoDB, Cassandra etc.. It would be great for the sales team.
Below is a simple representation of what it would look like.
SQL databases — Aspects with Configurable Datastore
It is possible to build a system like above ?
It could be configured at deployment time to work with say Postgres and Oracle for example.
It is SQL after all, how hard could it be ? Right ? Stop whining ! It is so very important from a “winning customers” perspective, Just do it !
Engineering view:
With many SQL databases, there are nuances in syntax.
Different DDL
For example at the the Oracle could we could
- Partition a table Based on partition keys
- And then within a partition we could have soft hash partitions on other keys. In effect each partition could have soft partitions within them, for efficient queries.
Now this is not needed for all use cases, but maybe needed for a certain use case as we would now get very good query performance with a little change in the DDL.
Oracle does allow us to auto-create partitions, which is such a nice feature.
Postgres on the other hand allows us to do the same things, in a different way. Hence we will have to do the same DDL in a different way.
Different DML
So now we know DDL is different between the two schemas.
Since DDL is different, it not not difficult to guess that DML is also different between the databases.
ORM — Object relational mapper
What is ORM ? read here
So what ? Would an ORM tool not fix the issue with multiple SQL databases? Just use an ORM tool!
Well yes and no, If you have simple CRUD type use case, then yes.
If you need something more than CRUD, it tends to get very complicated very fast. We have the added complexity of the way the ORM tool does non-CRUD queries, which will lock us to that ORM tool, an aspect we wanted to avoid in the first place.
Even after using the ORM tool we will have to perform certain aspects, like say “table partitions” in each database different way.
Hence having a configurable SQL DB layer will need more work, to code and maintain.
NoSQL databases — Aspects with Configurable Datastore
SQL databases have the ANSI SQL standard, at at the bare minimum.
NOSQL databases are of many kinds do not necessarily have a standard.
The MongoDb site has a good list here
Developers who come from SQL to NoSQL yearn for standards, look at this nice question and the answer is so good.
It is clear i guess, having a configurable NoSQL DB layer, will need effort to, code and maintain.
NewSQL databases — Aspects with Configurable Datastore
NewSQL db’s do seem to support SQL. Apache Ignite and VoltDB support SQL.
However the SQL support seems limited and does have nuances.
Look at Apache Ignite DML support, only Merge, Insert, Update and Delete are supported.
Ignite’s JDBC driver cannot just use a JDBC connection pool as it does not support transactions.
Ignite compute seems to be a good way to use it, which is not JDBC. This is perfectly ok as it does not need to be a SQL database.
VoltDB is similar.
Look at its SQL support.
This is perfectly ok as VoltDb does not claim to be a SQL database.
Hence having a configurable NewSQL DB layer will need effort to, code and maintain.
Aspects to consider
With a product that is fully configurable comes the following
Positives
- We can say things like, ‘Our product will work on any hardware and software you prefer to have’.
- Initially happy customers who can reuse their existing infrastructure, or at least justify purchase of a product internally.
Aspects to consider
- A lot of automation in product development — time and money.
- A lot of infrastructure in product development, to test all the combinations of software and hardware — time and money.
- A lot of support in product development, to get support for all the combinations of software and hardware — time and money.
Negative
- Unhappy customer if all of the above problems are not resolved.
Conclusion
What seems nice on paper may take cost, time and effort to develop and maintain.
Most of the time, software takes 10% time to develop and 90% time to run and maintain.
We need to be willing to spend the time and effort for the “90%”.
Questions ?
Lastly could we have all our software as a service, hosted in the cloud such that, clients can just use it and not need to invest in any hardware or software ?
Could we go one step further and use the serverless architecture ?
Why ?
Hardware and Server Software needs to be bought and maintained, what if we do not need to worry about any of that ?
Lastly an interesting read here from Martin Fowler on serverless architectures
