I've been writing software since 1979, floating along from Algol and Lisp in the classroom to Fortran in the research project, at Big Pharma, at Big Insura...until finally I landed a job at Oracle Corporation, found PL/SQL, wrote a book and settled into specializing on a single technology for the last 22 years.
And all along the way, I wrote code to satisfy the requirements of my end users, who were almost always employees of companies. In other words, not "retail."
But with the advent of the Internet and World Wide Web, and then especially with the viral use of smartphones and tablets, the predominant "end user" of applications is now, well, everyone.
But when everyone is a user, what are the apps for?
They used to help organizations meet their goals (make money, end hunger, above all sell things). And in this context, the data and the database were absolutely, fundamentally important.
When you received an order for a product, you really did want to be able to trust that the order information was saved and saved correctly. When you shipped the product, the inventory had to be updated properly. Otherwise chaos would ensue.
No one helped (or helps, today) avoid data chaos better than the Oracle Database.
Readers never block readers, writers never block readers.
Optimal performance without any sacrifices to data integrity.
All great stuff, and that's one of the reasons that Larry Ellison is one of the wealthiest people in the world: Oracle Database works.
Oh, and the SQL language was (and remains) a transformational way to work with data. And the PL/SQL language is a powerful extension to SQL, allowing developers to implement any business requirement imaginable.
But now....now....everyone's a user. And what exactly are the apps all these new users need?
Well, "need" is an interesting word, easily confused by humans with another four-letter word: want.
Look at the apps that dominate the end user space, that ushered in the era of Big Data, that led to the creation of new database technologies and even putting the word "No" in front of one the most successful software products ever (SQL).
Etc. etc. etc. So many of these websites and products give everyone a way to say:
"Look at me! I am here! This is what I am doing!"
"What are my high school buddies doing now?"
"Help! Emergency! I am bored! Please distract me on whatever screen is most accessible."
[And almost all of them are "free", funded by advertising. But I won't get started on that topic, right now.]
Who likes to be bored? That is, who likes to not be distracted, to be stuck inside your own brain, having to think about things, or even worse to pay deep attention to the world around you?
Apparently, not a whole lot of us.
And before I lose the train of thought/argument for this post, let's bring it back to data.
Very few humans have the privilege, if one could call it that, to be known, to be recognized, to be famous. But I think that all or many of us do crave that attention, in part because it is a validation of one's life, an injection of meaning. And we also like to make connections with other humans, which is harder and harder to do when you spend lots of time in front of a TV or video game deck or in a book.
And so when it became possible to have your own "wall" and build your own "timeline" that everyone, or at least your "friends" could see, there was no stopping us.
But think about the data: when you post an item about what kind of shampoo you used in your recent shower, it really doesn't matter if it takes a while for the info toy show up, or even if it is lost entirely in the ether. Sure, it might be frustrating, but you just shrug and post an item about what kind of beer you enjoyed for dinner. And that one sails through.
In other words, for many of our new everyone users with their new everyone apps, data integrity and consistency are not that critical. It's all about helping us avoid being bored, and that happens simply by typing our tweets, sharing our photos. Having it published is just the icing on the cake.
Combine that lack of concern about integrity with the enormous increase in volume when everyone becomes a user, and it's not hard to see why the beautiful features of SQL and Oracle Database are less critical - for this particular segment of software.
[And for the limited period of time needed for Oracle to improve its database technology to handle massively large data sets and high throughput.]
What is more difficult to see is why so many software developers and CIOs would conclude that data integrity and consistency is no longer needed for lots of other apps in which boredom does not drive the process.
But of course what goes around comes around. So now Google, the creator of Hadoop, has come to realize that it's really hard to build applications on top of that cool, new database. What's needed is a language to access and manipulate that data. Which means it's time for....wait for it...."NewSQL".
In the meantime, Oracle continues to strengthen its core database technology, extend it into new areas, boost performance by orders of magnitude (see the new In Memory database option).
In the meantime, we in the software industry need to help our End Users End their Boredom.
It's not going to be easy, and it is a never-ending task for sure, but at least it generates lots of data.