UnderGround Forums
 

ITGround >> database quasi rant/discussion for intermed/pros

| Share | Email | Subscribe | Check IPs

11/14/12 11:34 PM
Ignore | Quote | Vote Down | Vote Up
theseanster
359 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 5/13/02
Posts: 21223
 

A couple of useful bits of information to make you better than most of the "database experts" out there that really fail the logic test for design. Up front let me say that the following statements can be disregarded if you are inheriting an existing database for your software application, but when building from scratch, here's my take.

A lot of CS university courses will take an approach where you are given a set of business requirements from which you will derive a database design (complete with normal form to the nth degree) and graded on the design with perhaps a fet gotchas in the requirements along the way. If you've been victim to this type of course and I most certainly was in classes ranging from IMS, DB2, Access, and SQL Server, fear not....this is in almost every case complete and utter bullshit.

Yeah...that's right...I said it. First of all, business requirements should be used to derive USE CASES, which in software terms translates to Business Interface (or Facade layers for you old bastards) object models and function signatures. Database designs are really supposed to be a product of the best way to STORE data and this is not always in harmony with a use case. In fact, I'd argue that in most cases it is not. Now I realize the popularity of ORMs, but forget about those for a minute because those are IMHO abstractions for developers who are lazy, under huge pressure to meet a deadline, have weak SQL skills, or perhaps even all three. Vote me down if you want, it is true. 

Now, what do I mean by the best way to store data? It means: it depends. How should you design your database then? It depends. If I'm building a data warehouse that ultimately I want to compile into an OLAP cube, I'm going to denormalize and follow a basic star schema (or snowflake if I have dimensions that depend on other dimensions such as Person being sliced by Ethnicity, Age group, etc). If I'm building a company interanet Web application, again it depends on the usage. Denormalize for read-intensive apps, normalize for heavy insert-update-deletes....but at the end of the day I'm not going to build a database from business requirements.

OK so who gives a shit? Well you should! This database design first approach is exactly the root of all evil I'm trying to get people to avoid on a regular basis. It is the same backwards thinking that gets people to write their application code by developing an API without a use for it. Yes, THAT IS BAD!

You don't design an abstraction without at least two concrete implementations. What does that mean? For those of you who have taken a programming course before and have seen the old polymorphism example of creating an abstract class called SHAPE which then has derived classes CIRCLE, SQUARE, TRIANGLE and you assign various circles, squares, and triangles into an array of shapes and then iterate the array to call shape.Draw().....let me ask you a question. How would you know how to properly design the Shape class without having the concrete circle, square, and triangle? Well...because your CS instructor told you to...but that's not realistic when developing software. Abstractions are created based on a need, and that generally comes from either refactoring or while designing a solution where you find an abstraction is needed. Same thing with a database!

To further explain, when developing business logic in your applications, how many times have you ever stopped and said "damn I wish this database had another column which was a flag that let me know some state value of this situation?" If you're not a programmer, then never, but trust me it happens all of the time when people try to design backend to front end.

When developing an API, it is ALWAYS (yeah that's usaully a bad word to look for in a T/F quiz but I don't care) better to design based on writing the client CONSUMER code for the API to get a vision of usability. In other words: front to back.

With a software application, you gather business requirements, build a conceptual model of subsystems, possibly the use case layer which is your Business Interface and then start by fleshing out requirements with a UI prototype. If the prototype gets user approval, then you can build the logic behind it which includes your API, which then does CRUD operations against your backend database. This design is iterative and that's OK.

FRAT version: design front to back every time you possibly can.

11/17/12 1:41 AM
Ignore | Quote | Vote Down | Vote Up
NinjaNate
Send Private Message Add Comment To Profile

Member Since: 4/24/02
Posts: 2135
Nice to see a development related post. To play devil's advocate here - in the enterprise world, like banking for example, often times APIs come and go but the database lasts for years and years so its critical to get its design right.
11/17/12 11:31 AM
Ignore | Quote | Vote Down | Vote Up
theseanster
359 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 5/13/02
Posts: 21246
NinjaNate - Nice to see a development related post. To play devil's advocate here - in the enterprise world, like banking for example, often times APIs come and go but the database lasts for years and years so its critical to get its design right.

True, there are definitely exceptions. I've done gigs at airlines where their API was basically a screenscrape for old systems. Other than that system or something I've inherited as legacy or 3rd party, most of your typical intranet sites or even internet sites should be developed front to back. Especially these days when management is too lazy to provide detailed requirements or give you time to create a detailed design (don't even get me started on other things like QA, that's what we have customers for right???)

4/18/13 12:25 PM
Ignore | Quote | Vote Down | Vote Up
chew22
21 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 2/28/05
Posts: 855
In today's IT, 4th normal DB designs are pretty much obsolete. NoSQL solutions are becoming more popular, as well as blob storage.

The reality is, how much do designs REALLY need an RDBMS? What is it about a particular solution that REQUIRES relational interaction? Even if it does, why does that requirement need to be fulfilled in the DB, rather than the domain layer? RDMS design basically forces things into a certain pattern.
4/18/13 12:30 PM
Ignore | Quote | Vote Down | Vote Up
reagan123
175 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 1/1/01
Posts: 3731
S Phone Post
4/19/13 11:51 AM
Ignore | Quote | Vote Down | Vote Up
BigWilliam
372 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 3/7/05
Posts: 20996
NoSQL is not new, databases were designed that way in the 1970's out of necessity for storage space. That idea was abandonded because of the functionality an relation system offers. Having the record layouts specified and protected only in application code is dangerous and foolish unless you have some real need to process billions of records at a time. And most people do not.
4/22/13 7:43 PM
Ignore | Quote | Vote Down | Vote Up
chew22
21 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 2/28/05
Posts: 856
LOL @ "dangerous and foolish". Bravo ancient COBOL / DB2 coder, bravo ....

Check out a few of the titles in Martin Fowler's library on this topic here: http://www.martinfowler.com/articles/enterprisePatterns.html

There's quite a few more than one way to design an app, product, model, etc. NoSQL is currently swinging back into popularity because it's cheaper for cloud storage than a RDB. And the Domain Model is one of Fowler's architecture patterns which does exactly that dangerous and foolish stuff of driving business logic to a layer.

Of course there are always those that require their business logic to be managed in stored procedures.


4/24/13 11:08 AM
Ignore | Quote | Vote Down | Vote Up
BigWilliam
372 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 3/7/05
Posts: 21058
Heh, I am not ancient, I don't write COBOL and you don't 'code' DB2. I am experienced and I do know the value of RI in protecting data from bad programmers that think they should be able to carve records up any way they can. NoSQL is useful for Google and Amazon because they have need to process billions of records on the fly. Using it for day to day stuff in shops where you don't have that need is pointless. You realize Fowler is a salesman trying to sell a product right?
4/24/13 3:11 PM
Ignore | Quote | Vote Down | Vote Up
Titan Ape
44 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 5/17/09
Posts: 376
I have no dog in this fight, but I thought this article was interesting.

https://moot.it/blog/technology/redis-as-primary-datastore-wtf.html

The tl/dr version: We use nosql as our main data store because our use case requires it.
5/3/13 6:18 PM
Ignore | Quote | Vote Down | Vote Up
chew22
21 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 2/28/05
Posts: 868
NoSQL is useful wherever you really don't need a table join. It is a different approach, not Fowler trying to sell something. I just love your clichés:

"bad programmers who think they should be able to carve records up any way they can".

You do realize that a "record" is a mental construct made by man, don't you? It isn't a higher life form with its own bill of rights.

Oh and kudos for making one of my arguments for me. Google and Amazon use NoSQL to "process billions of records on the fly". So in other words I hear a concession that NoSQL is more performant than RDBMS for large scale data operations.
8/2/13 1:47 AM
Ignore | Quote | Vote Down | Vote Up
#justice4babyTRAYtray
359 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 5/13/02
Posts: 22140

Forgive my lack of NoSQL experience outside of something like Lucine, but how exactly is a "hash" operation in NoSQL any faster than a clustered index lookup in something like SQL Server? I thought NoSQL was best for non-structured data or information such as documents.

8/16/13 12:10 AM
Ignore | Quote | Vote Down | Vote Up
OneScoup
Send Private Message Add Comment To Profile

Member Since: 4/18/02
Posts: 22214
I'm not going to take sides in the NoSQL debate but Fowler is the man. His books are nearly modern day classics in design patterns and architecture.
8/16/13 1:21 AM
Ignore | Quote | Vote Down | Vote Up
#justice4babyTRAYtray
359 The total sum of your votes up and votes down Send Private Message Add Comment To Profile

Member Since: 5/13/02
Posts: 22228
Definitely. GoF is timeless Phone Post 3.0

| Share | Email | Subscribe | Check IPs

Reply Post

You must log in to post a reply. Click here to login.