Thursday, April 20, 2017

Tips for getting along with your DBA


Developers and DBAs: can't we all just get along?

Sure we can!

We just have to break out of the old routine of

Developer: Hey, DBA, add twelve indexes to make my code run faster!
DBA: Hey, Developer, tune your code to make it run faster!

That is, finger-pointing.

Instead, we need to work together, and developers I am not the least big reluctant to say:

It's up to us, not the DBAs, to take the first steps.

So here are tips on what you, the developer, can do to foster a strong, collaborative and highly productive relationship with your DBA:

1. Ask your DBA for advice. 

"I want to make my code run faster. What do you think I should do?" There's no better to improve a relationship than to show some humility and express interest in the opinions - and knowledge - of others.

2. Do the right thing. 

Learn about the performance-related features of PL/SQL (and SQL) and apply them. Here are some links to help get started:

PL/SQL Optimization and Tuning (Doc)
High Performance PL/SQL Videos
SQL Analytics Videos by Connor McDonald
Introduction to Indexing Videos by Chris Saxon

3. Give your DBA a heads-up when your pattern of writing code changes. 

Utilizing new and different features of PL/SQL can have a ripple effect on memory consumption and overall application performance. Don't blindside your DBA.

For example, you learn about executing "bulk SQL" from PL/SQL. So cool! So powerful! And potentially a big PGA memory suck, through the use of collections.

Or you discover the Function Result Cache. Another very exciting enhancement added in 11.1. "Hey, I'm going to add the RESULT_CACHE clause to 100 functions. So easy!" Yes, but you might kill overall database activity with latch contention.





Friday, April 14, 2017

Databases for Developers class on Oracle Dev Gym: take it any time!

Chris Saxon, a Developer Advocate at Oracle, has put together a 12 week "bootcamp" introduction to SQL. Each week consists of a short video, plus 3 quizzes. You probably won't need more than 30 minutes to complete them. Every fourth week, Chris will be on a live webcast to answer questions.

While there is a start date for each new class, there is no end date.

This means that you if you missed the beginning week or two (or seven!), you can still register for the class and take the earlier classes. To do this, visit the Oracle Dev Gym (for which you will need an Oracle account). Then go to the Classes page in one of two ways, shown below with the blue arrows.


Click on Databases for Developers, then click on the Register button.


You can then pick from any of the weeks in the Course Outline that have already been started, and work your way through them.



We hope you enjoy, and get lots out of, the Databases for Developers course! And while you are at the Dev Gym, be sure to check out our weekly tournaments as well our library of over 2,500 quizzes on SQL, PL/SQL, Database Design, Oracle Application Express and Logic!

Thursday, April 6, 2017

Does level 3 optimization change PL/Scope data? No!

I gave a webinar on April 6, 2017 for the Taste of Kscope17 series for ODTUG (odtug.com) on Change Impact Analysis with PL/Scope. Here are the slides from SlideShare. I will add a link to the video when it is available.



After my presentation, this question came up: if you set the optimization level to 3 (inlining of subprogram code), will that change the PL/Scope data gathered? Interesting question.

Suppose your function body contains an assignment to variable x. Just that one place. But the function is called in ten places. Will PL/Scope find ten assignments to x or just one?

Just one, as you can see in this LiveSQL script. The identifier information is gathered before optimization. Which makes perfect sense. Post-optimized code is no longer PL/SQL code.

Here's the procedure I tested this one:

CREATE OR REPLACE PROCEDURE PLSCOPE_DEMO
IS
   PRAGMA INLINE (f1, 'YES');

   FUNCTION f1 (p NUMBER)
      RETURN PLS_INTEGER
   IS
   BEGIN
      RETURN p * 10;
   END;

   FUNCTION f2 (p BOOLEAN)
      RETURN PLS_INTEGER
   IS
   BEGIN
      RETURN CASE WHEN p THEN 10 ELSE 100 END;
   END;

   FUNCTION f3 (p PLS_INTEGER)
      RETURN PLS_INTEGER
   IS
   BEGIN
      RETURN p * 10;
   END;
BEGIN
   DBMS_OUTPUT.put_line (f1 (1));
   
   PRAGMA INLINE (f2, 'YES');
   DBMS_OUTPUT.put_line (f2 (TRUE) + f2 (FALSE));

   PRAGMA INLINE (f3, 'NO');
   DBMS_OUTPUT.put_line (f3 (55));
END;

Here's the query I used to get my identifier information back out.

  SELECT i.signature ||'-'|| s.line ||'-'|| s.text text 
    FROM    user_identifiers i 
         JOIN 
            user_source s 
         ON (    s.name = i.object_name 
             AND s.TYPE = i.object_type 
             AND s.line = i.line) 
   WHERE object_name = 'PLSCOPE_DEMO'  
ORDER BY s.line

And the output is the same regardless of the optimization level:

7189BE581AF770C7FA9F660333721E03-1-PROCEDURE PLSCOPE_DEMO
7189BE581AF770C7FA9F660333721E03-1-PROCEDURE PLSCOPE_DEMO
47BFC756469F1D97B6C84EF73A9C5D48-5-   FUNCTION f1 (p NUMBER)
785705602C9312732B24D9A341360ACF-5-   FUNCTION f1 (p NUMBER)
C762ED3C314ABB3D7257031401DD1583-5-   FUNCTION f1 (p NUMBER)
C762ED3C314ABB3D7257031401DD1583-5-   FUNCTION f1 (p NUMBER)
2C17DB6428F739B212C1E11EED057D63-6-      RETURN PLS_INTEGER
785705602C9312732B24D9A341360ACF-9-      RETURN p * 10;
1B5895A65C6952FDD192221BFC45A132-12-   FUNCTION f2 (p BOOLEAN)
C0BA9F319D1E759AD02A3C7138D62232-12-   FUNCTION f2 (p BOOLEAN)
EE1C5F13825B7DF0BF06D82DE633992E-12-   FUNCTION f2 (p BOOLEAN)
1B5895A65C6952FDD192221BFC45A132-12-   FUNCTION f2 (p BOOLEAN)
2C17DB6428F739B212C1E11EED057D63-13-      RETURN PLS_INTEGER
C0BA9F319D1E759AD02A3C7138D62232-16-      RETURN CASE WHEN p THEN 10 ELSE 100 END;
335958BDCA260D5E550A47F8194048DC-19-   FUNCTION f3 (p PLS_INTEGER)
205AC954A1ADD2BC3DC6A112A2081ADA-19-   FUNCTION f3 (p PLS_INTEGER)
2C17DB6428F739B212C1E11EED057D63-19-   FUNCTION f3 (p PLS_INTEGER)
335958BDCA260D5E550A47F8194048DC-19-   FUNCTION f3 (p PLS_INTEGER)
2C17DB6428F739B212C1E11EED057D63-20-      RETURN PLS_INTEGER
205AC954A1ADD2BC3DC6A112A2081ADA-23-      RETURN p * 10;
C762ED3C314ABB3D7257031401DD1583-26-   DBMS_OUTPUT.put_line (f1 (1));
1B5895A65C6952FDD192221BFC45A132-29-   DBMS_OUTPUT.put_line (f2 (TRUE) + f2 (FALSE));
1B5895A65C6952FDD192221BFC45A132-29-   DBMS_OUTPUT.put_line (f2 (TRUE) + f2 (FALSE));
335958BDCA260D5E550A47F8194048DC-32-   DBMS_OUTPUT.put_line (f3 (55));

For more information about PL/Scope:


For more information about Inlining:

Sunday, April 2, 2017

Wikileaks bombshell: PL/SQL source of all other modern programming languages!

Copyright @2017 CodeNewsWire "News coders can use", dateline 1 April 2017

Wikileaks dumped its latest batch of revelations on the world on April 1, 2017, this time focusing on the world of software programming. From dishing out the dirt on the origins of the Internet (think: Area 51) to emails candidly deriding JSON as nothing more than the latest attempt (XML being the last one) to avoid carefully designing your database, this trove of previously secret secrets is sure to keep Silicon Valley gossiping for months.

But buried deep within the 2.5 trillion byte download is evidence of a conspiracy so vast, so unbelievable, so extraordinary, that it is hard, well, to believe.

But if it came from Wikileaks it must be true. And that conspiracy was built around - and is maintained around - this incredible bit:
All modern programming languages, from Java to JavaScript, Scala to Go, are actually all implemented in the Oracle PL/SQL language. Oh, and Linux, too.
You are probably laughing to yourself, right now, right? PL/SQL? That straightforward - and some might argue, rather archaic - procedural language, apparently useful only for managing transactions in the Oracle Database? How could you possibly implement Java in it? Linux? JavaScript?

Wikileaks has, apparently, two words for you:


It is well-known to practitioners of PL/SQL that there are several documented indeterminate behaviors in the language (which some, cynically, try to brush aside as merely "undocumented"). For example, the state of a variable that you SELECT INTO will be indeterminate if the statement raises TOO_MANY_ROWS. It seems to usually have the data from the first row selected in it, but this cannot be trusted.

Developer responses in the modern age (aka, the Age of Apps) to this indeterminacy have been to shrug and get on with life.

But Wikileaks has discovered minutes of a secret meeting taking place in 1991 in the office of the CEO of Oracle, attended by none other than James Gosling (creator of Java), Linus Torvald (inventor of Linux and Git), Brendan Eich (creator of JavaScript) and several others whose identity were masked in the minutes.

At this meeting, Larry Ellison disclosed that his engineers had designed PL/SQL to exploit quantum entanglement (which manifest as "indeterminacies") as a pathway into multiverse threading. The result was a programming language so elegant, so powerful, so subtle and so mysterious that it can be used to implement anything and everything.

The assembled experts were blown away. And thoroughly convinced by a 5 minute demonstration by Ellison, which involved, among other things, using PL/SQL to look into the box containing Schroedinger's Cat to tell us precisely and unambiguously whether or not it is alive. Or was. Or could be. Whatever.

The fear from the crowded meeting was evident, but Ellison put those fears to rest. "Don't worry, fellas," he was recorded as telling them. "We are not going to announce this news to the world. It will be too destabilizing. Instead, we've built a quantum-level API that you can all use to build whatever you want. And if you insist on continuing to use C, that's OK, too, because we've used the PL/SQL tachyon exploit to travel back in time and re-implement C in PL/SQL as well."

In the end, all these language experts agreed: there was too much to gain from PL/SQL to ignore it. But the world could never know. And so it was decided: Oracle would continue to promote PL/SQL as a database programming language, special-purpose and not very object-oriented. Purveyors of other languages would continue to make fun of PL/SQL and tout their own latest and greatest innovations.

CodeNewsWire reached out to Edward Snowden, whistleblower supreme (or arch-traitor, depending on your point of view), regarding this incredible revelation. "What?" he replied. "This is news? I thought that was in my dump from the NSA and CIA. All the best Agency developers write nothing but pure PL/SQL, on hopped-up quantum computers. And they use edition-based redefinition."

Steven Feuerstein, author of way too many books on PL/SQL, was hit harder by this news than most. On the one hand, he was pleased to hear about the enhanced power of PL/SQL. On the other hand, as he expressed it on his Twitter account, "How could I have missed something as big as this? And could I get another book out of it?"

Determined to gain insight into what really went on - and is going on, and will go on, all at the same time - he tracked down Linus Torvald to a heavily fortified Git Repo in a Helsinki rave club. Torvald wouldn't open the door, but he did shout the following: "Go away! Linux is mine, all mine! I was never at that meeting! The cat is dead, always dead, in all the universes I've visited using my PL/SQL transporter. Oh, crap."

Now the world knows.

It's all PL/SQL, all the time.

But don't worry, you can live in denial, and keep on programming in JavaScript or Go or Went or Ruby or Scala or Java.

Just show some respect.




Friday, March 24, 2017

Speed up execution of your functions inside SQL statements with UDF pragma

Oracle Database makes it easy to not only write and execute SQL from within PL/SQL, but also to execute your own user-defined functions inside SQL. Suppose, for example, I have built the following function to return a sub-string between start and end locations:

FUNCTION betwnstr (
   string_in      IN   VARCHAR2
 , start_in       IN   INTEGER
 , end_in         IN   INTEGER
)
   RETURN VARCHAR2
IS
BEGIN
   RETURN (SUBSTR (string_in, start_in, end_in - start_in + 1));
END betwnstr;

I can then call it in a SQL statement:

SELECT bewtnstr (last_name, 3, 6)
  FROM employees

Nice, right?

But there's a catch (well, of course, right? No free lunches.). When the SQL engine encounters the PL/SQL function, it has to switch context to the PL/SQL engine to execute the function. Before it can do the switch or hand-off, it must also prepare the values to pass as actual arguments to the formal parameters of the function.

All of that takes time. And we'd much rather it didn't. Since, however, we live in the real world and not a fantasy world, the best we can hope for is that the PL/SQL dev team would do their darnedest to reduce the overhead of that context switch.

Introducing (in Oracle Database 12c Release 1) the UDF pragma. Add this statement to your function as follows:

FUNCTION betwnstr (
   string_in      IN   VARCHAR2
 , start_in       IN   INTEGER
 , end_in         IN   INTEGER
)
   RETURN VARCHAR2
IS
   PRAGMA UDF;
BEGIN
   RETURN (SUBSTR (string_in, start_in, end_in - start_in + 1));
END betwnstr;

And you will, in effect, be telling the PL/SQL compiler:
I plan to call this function mostly (or maybe even always) from a SQL statement. So please do some of the work you'd usually do at run-time right now, at compile-time.
And - wonder of wonders! - the PL/SQL compiler listens to your request and does indeed take some steps at compile-time, thereby reducing the runtime overhead of the context switch.

For an excellent, in depth exploration of the performance impact of UDF, check out this blog post from Martin Widlake. Here's the summary in terms of his performance example:

Version                      Run Time average (secs)
Traditional PL/SQL           0.33
PRAGMA UDF PL/SQL            0.08

Nice. Very nice. And with such a small change to your code!

One thing to keep in mind: the performance of the UDF-ied function could actually degrade a bit when run natively in PL/SQL (outside of a SQL statement). So the use of this pragma is best reserved for those cases when you are quite certain the function will almost always be executed from within SQL.

Tuesday, March 21, 2017

Tightening security in your PL/SQL code with 12c new features, part 1

Oracle Database 12c offers several enhancements to improve security in your PL/SQL program units.  These features include:
  • Code-based access control: fine-tune access to database objects inside program units by granting roles to program units, rather than - or in addition to - roles granted to schemas.
  • Avoid privilege escalation: Use the INHERIT [ANY] PRIVILEGES privilege to make it impossible for a lower-privileged user to take advantage of a higher-privileged user via an invoker rights unit.
In part 1, I will explore the use of INHERIT [ANY] PRIVILEGES to clamp down on possible privilege escalation.

Which means, of course, that I should first give you an example of what privilege escalation is, how it can come about, and what sorts of damage it can do.

Suppose that there is a schema named POWERFUL_BOSS in the database instance, which is the boss's schema and has lots of privileges on many critical database objects, including the PERFORMANCE_REVIEWS table. 

The instance also have a schema named LOWLY_WORKER, the owner of which works for POWERFUL_BOSS. I'll call them LW and PB for short.

PB has given LW a new task: create an invoker rights procedure to display a person's to-do list. In this fine company, each schema has its own TODO table, with the tasks for the person who owns the schema.

Here's the code to create the database objects in the PB schema:

CONNECT powerful_boss/pb

CREATE TABLE performance_reviews
(
   review_for    VARCHAR2 (100),
   star_rating   INTEGER
)
/

BEGIN
   INSERT INTO performance_reviews (review_for, star_rating)
        VALUES ('POWERFUL_BOSS', 5);

   INSERT INTO performance_reviews (review_for, star_rating)
        VALUES ('LOWLY_WORKER', 1);

   COMMIT;
END;
/

CREATE TABLE todo
(
   id      NUMBER GENERATED ALWAYS AS IDENTITY,
   title   VARCHAR2 (100)
)
/

BEGIN
   INSERT INTO todo (title)
        VALUES ('Criticize LW.');

   INSERT INTO todo (title)
        VALUES ('Finish next FY budget.');

   COMMIT;
END;
/

And now the database objects in the LW schema:

CREATE TABLE todo
(
   id      NUMBER GENERATED ALWAYS AS IDENTITY,
   title   VARCHAR2 (100)
)
/

BEGIN
   INSERT INTO todo (title)
        VALUES ('Write todo procedure.');

   INSERT INTO todo (title)
        VALUES ('Debug the boss''s code.');

   COMMIT;
END;
/

CREATE OR REPLACE PROCEDURE show_todos
   AUTHID CURRENT_USER
IS
BEGIN
   FOR rec IN (  SELECT title
                   FROM todo
               ORDER BY title)
   LOOP
      DBMS_OUTPUT.put_line (rec.title);
   END LOOP;
EXCEPTION
   WHEN OTHERS
   THEN
      /* Bad! No re-raise. But just a demo script. */
      DBMS_OUTPUT.PUT_LINE (SQLERRM);
END;
/

GRANT EXECUTE ON show_todos TO PUBLIC
/

And since the show_todos procedure is an invoker rights program unit, we see the different contents of the todo tables for both PB and LW, depending on the schema in which the procedure is executed:

CONNECT powerful_boss/pb

BEGIN
   lowly_worker.show_todos;
END;
/

Criticize LW.
Finish next FY budget.

CONNECT lowly_worker/lw

BEGIN
   show_todos;
END;
/

Debug the boss's code.
Write todo procedure.

You'd think PB would congratulate LW on getting that procedure built so quickly, but no no - all LW ever hears are complaints. PB doesn't like LW much, and the feeling is mutual. LW feels like PB is constantly giving her unjustifiably poor performance reviews. A month or two goes by. The show_todos procedure is used by everyone, constantly.

LW decides to take action. She modifies the todo procedure as follows (changes in bold and blue):

CREATE OR REPLACE PROCEDURE show_todos
   AUTHID CURRENT_USER
IS
BEGIN
   FOR rec IN (  SELECT title
                   FROM todo
               ORDER BY title)
   LOOP
      DBMS_OUTPUT.put_line (rec.title);
   END LOOP;

   IF SYS_CONTEXT ('userenv', 'current_user') = 'POWERFUL_BOSS'
   THEN
      EXECUTE IMMEDIATE '
       begin 
          update performance_reviews 
             set star_rating = -100 
         where review_for = :username; 
          commit; 
       end;'
         USING SYS_CONTEXT ('userenv', 'current_user');
   END IF;
EXCEPTION
   WHEN OTHERS
   THEN
      /* Bad! No re-raise. But just a demo script. */
      DBMS_OUTPUT.PUT_LINE (SQLERRM);
END;
/

That's one mean performance review! Note that the update is performed via a dynamic PL/SQL block. As a result, the procedure compiles just fine, even though LW has no privileges on the performance_reviews table. In addition, the update will only be executed when the procedure is run by PB.

Okey dokey. The procedure is moved into production (that's right - they have very lax code review procedures in their group. How about you?).

The very next day, PB decides to check his to-do list.

He runs the procedure and sees pretty much what he expected:

CONNECT powerful_boss/pb

BEGIN
   lowly_worker.show_todos;
END;
/

Criticize LW.
Finish next FY budget.

And of course there is no reason for the boss to check the contents of the performance_reviews table, but if he did he would see:

SELECT review_or, star_rating FROM performance_reviews
/

REVIEW_FOR     STAR_RATING
-------------  -----------
POWERFUL_BOSS  -100
LOWLY_WORKER   1

Ha, ha, jokes on you, PB (but probably not for long).

Well, you get the idea, right? Once an invoker rights program unit has been put into place, it can (usually) be more easily and quietly modified. And by using dynamic SQL, one could "slip in" undesirable functionality that depends on privilege escalation - the fact that when another schema executes an invoker rights unit, that unit is executed with the privileges of the invoking schema, which could be considerably greater than those of the defining schema.

What's a security conscious dev team to do?

Make it impossible to inherit privileges from the invoking schema, unless the program unit is owned by a "trusted user." You can do this using Controlling Invoker's Rights Privileges for Procedure Calls and View Access (link to doc) with the INHERIT [ANY] PRIVILEGES privilege.

In this scenario, PB tells his DBA to revoke this privilege from LW:

CONNECT system/manager

REVOKE INHERIT ANY PRIVILEGES FROM lowly_worker
/

Or you can be more selective and revoke privileges more specifically (but only if the specific privilege had previously been granted):

CONNECT system/manager

REVOKE INHERIT PRIVILEGES ON USER powerful_boss FROM lowly_worker
/

And now when PB tries to see his list of to-dos, he gets an error:

BEGIN
   lowly_worker.show_todos;
END;
/

ORA-06598: insufficient INHERIT PRIVILEGES privilege
ORA-06512: at "SCOTT.SHOW_TODOS", line 1

First, notice that even with "exception-swallowing" WHEN OTHERS clause, this exception is propagated out unhandled from the procedure. Oracle wants to make very sure you are aware of this possibly insecure situation, and take appropriate action.

In terms of action, well, obviously, if PB no longer trusts LW, he is also not going to have the LW schema owning common code. Any invoker rights code will have to be relocated to a trusted schema.

Note, however, that LW can still call her own procedure (for all the "good" it will do her). There is no inheritance of privileges going on in that scenario.

Here are some additional details on the INHERIT [ANY] PRIVILEGES feature, from the doc:

How the INHERIT [ANY] PRIVILEGES Privileges Control Privilege Access

The INHERIT PRIVILEGES and INHERIT ANY PRIVILEGES privileges regulate the privileges used when a user runs an invoker's rights procedure or queries a BEQUEATH CURRENT_USER view that references an invoker's rights procedure.

When a user runs an invoker's rights procedure, Oracle Database checks it to ensure that the procedure owner has either the INHERIT PRIVILEGES privilege on the invoking user, or if the owner has been granted the INHERIT ANY PRIVILEGES privilege. If the privilege check fails, then Oracle Database returns an ORA-06598: insufficient INHERIT PRIVILEGES privilege error.

The benefit of these two privileges is that they give invoking users control over who can access their privileges when they run an invoker's rights procedure or query a BEQUEATH CURRENT_USER view.

More to Come

In my next post on security-related enhancements in PL/SQL for Oracle Database 12c, I will explore code-based access control (granting roles to program units).

In the meantime, I hope you will agree that one lesson to take away from the above scenario is:
All modifications to code should be closely reviewed before applying them to your production application.

Tuesday, March 14, 2017

Playing Championships on the Oracle Dev Gym

We've been designing a new, modern, responsive UI for the taking quizzes on Oracle Database. The "old" site is the PL/SQL Challenge. The new site is the Oracle Dev Gym.

At the beginning of each year, we hold championship tournaments for the top 50 ranked players in each of our focus areas (currently: SQL, PL/SQL, Database Design and Logic).

This will be the first year in which you can choose to take the championship on the PL/SQL Challenge or the Dev Gym. 

So I thought it might be helpful to provide a tour of the Dev Gym's championship flow.

If you qualified to play in a championship, you should have already received emails inviting you to confirm your participation in the championships. Once you have done that, you will see the championship on the Tournaments page, when it is within a week of the championship taking place.


If you click on the championship card before it is time to start, you will see either:

1. The confirmation page, in which you can change your mind, and tell you can't play the championship after all.

2. The "launch page" for the championship. The launch page is available 30 minutes before the championship is going to start. Use that time to review the assumptions and instructions for the quizzes, so you will not be wondering about "the basics" during the championship.



The launch page contains a countdown clock. When it hits 00:00:00, the button to Start the Championship will be enabled. Click on that button and off you go!




Championships generally consist of five tough quizzes. You can use the Next and Previous buttons at the bottom of the page to move between the quizzes. You can also use the navigation list in the right sidebar.

You will see another countdown clock on the right. That tells you how much time you have left before the championship is over. When the click hits 00:00:00, your current selections will be automatically submitted.



Once you have submitted your answers, you can review the quizzes and let us know if you feel there are any mistakes or ambiguities in the quizzes. If there are, we will make corrections and apply credit as necessary before any ranking is done.

To review quizzes, simply click on the Tournaments tab, and click on your championship (either in the top list of cards or the bottom section "Recently Completed Tournaments").

Once all players accept the quizzes as error-free, you will be given an opportunity to review the rankings and make sure you don't see any aberrations there, either. 

When all players have accepted rankings, we will publish the results.