You might think , that i got possessed with baselines and profiles but I like exploring them.

The deeper I explore, the more fun it is.

In the last post, I had covered how to convert profiles baselines into profiles.

Taking a step further this time, I tried to sync my explain plan with execution plan.

A bit of history:-

Some days back, I ran into issue where explain plan was looking very much optimal but when I executed the query, it was picking different plan.

So, i thought why not get a way to force the plan of explain plan during execution.

In other words, why not sync my explain plan and execution plan.

So, i started exploring plan_table and voila….there it was. My favorite column other_xml.

By now, you must have realized, that oracle stores all the hints and some other stuff like (plan_hash2->This i will explain in some other post) in this column. When you use dbms_xplan, oracle parses this column and print in readable format.

I too extracted the info in my desired format from this column and create a script(xplanprofile.sql) to generate profile.

xplanprofile.sql

Session1> explain plan for select * from master_log where task_id=:b1;
Session1>@xplanprofile
enter original sql_id:- 9ghy678ujy345
enter bad plan hash value to be tuned :- 3709384
'===================OUTPUT========================='

declare
ar_profile_hints sys.sqlprof_attr;
begin
 ar_profile_hints := sys.sqlprof_attr(
 'BEGIN_OUTLINE_DATA',
 'INDEX_RS_ASC(@"SEL$1" "MASTER_LOG"@"SEL$1" ("MASTER_LOG"."TASK_ID"))',
 'OUTLINE_LEAF(@"SEL$1")',
 'OPT_PARAM(''optimizer_index_caching'' 80)',
 'OPT_PARAM(''optimizer_index_cost_adj'' 1)',
 'OPT_PARAM(''_optimizer_connect_by_cost_based'' ''false'')',
 'OPT_PARAM(''_optim_peek_user_binds'' ''false'')',
 'OPT_PARAM(''_b_tree_bitmap_plans'' ''false'')',
 'DB_VERSION(''11.2.0.2'')',
 'OPTIMIZER_FEATURES_ENABLE(''11.1.0.7'')',
 'IGNORE_OPTIM_EMBEDDED_HINTS',
 'END_OUTLINE_DATA'
);
for sql_rec in (
 select t.sql_id, t.sql_text
 from dba_hist_sqltext t, dba_hist_sql_plan p
 where t.sql_id = p.sql_id and p.sql_id = 'sehy678ujy345' and p.plan_hash_value = 3709384 and p.parent_id is null
 ) loop
 DBMS_SQLTUNE.IMPORT_SQL_PROFILE(
 sql_text => sql_rec.sql_text,profile => ar_profile_hints,name => 'PROFILE_'||sql_rec.sql_id
 );
 end loop;
end;
/

In my last post, we have discussed various workarounds to fix the execution plan of the query if its not picking baseline.

Of the given 3 methods, I was interested in the 3rd one. What got me interested is the idea if we can create profiles from  baselines. 

The next obvious question is “why you need to do so”.

The 2 reasons are

1. Baselines not getting used. In this case you can create a profile out of baseline and then impose oracle to use this profile.

2. In 11g, we have ADG (active standby database). Guess what!!! Baselines doesn’t work in ADG. So if you fix query on primary, it won’t pick the plan in standby. So you better create a profile.  Also you may need to bounce your standby in order for profile to get used (because of another bug).

Before I talk about script to convert baselines into profiles, lets step back and think whats is that thing which is common in baselines and profiles(if at all there is) and what is that thing which makes them different.

Both baselines and profiles store data in form of hints.

Only difference is in case of baselines oracle tries to regenerate the same plan using hints and if it can’t then it will not use the baseline.

However, in case of profile, oracle will use the hints to generate the execution plan which it will use. Therefore, there is no guarantee that oracle will land on same plan again and again and hence plan flip can occur.

I will soon write about hints and how oracle stores them for a given plan.

Anyway, coming back to our point, oracle stores the execution plan in form of hints. Therefore, for each plan it captures or generates, it must store them somewhere. The idea is to get those hints and create the profile.

planprofile.sql:-

Session1>select plan_name from dba_sql_plan_baselines where rownum < 2;

PLAN_NAME
------------------------------
SYS_SQL_PLAN_086da2eaeec7bdef

1 row selected.

Session1>@planprofile
enter sql_id to be fixed:- dgh672sad89j4
baseline plan name:- SYS_SQL_PLAN_086da2eaeec7bdef
enter bad plan hash value to be tuned :- 12345678
'===================OUTPUT========================='

declare
ar_profile_hints sys.sqlprof_attr;
begin
 ar_profile_hints := sys.sqlprof_attr(
 'BEGIN_OUTLINE_DATA',
 'FULL(@"SEL$4" "MRH"@"SEL$4")',
 'NO_ACCESS(@"SEL$3" "from$_subquery$_007"@"SEL$3")',
 'FULL(@"SEL$6" "MRH"@"SEL$6")',
 'NO_ACCESS(@"SEL$5" "from$_subquery$_004"@"SEL$5")',
 'PQ_DISTRIBUTE(@"SEL$F5BB74E1" "RMS"@"SEL$2" NONE BROADCAST)',
 'USE_NL(@"SEL$F5BB74E1" "RMS"@"SEL$2")',
 'LEADING(@"SEL$F5BB74E1" "H"@"SEL$1" "RMS"@"SEL$2")',
 'INDEX_RS_ASC(@"SEL$F5BB74E1" "RMS"@"SEL$2" ("METRICS_SUMMARY95"."MVIEW_NAME" "METRICS_SUMMARY95"."OWNER"))',
 'NO_ACCESS(@"SEL$F5BB74E1" "H"@"SEL$1")',
 'PQ_DISTRIBUTE(@"MRG$1" "H"@"MRG$1" BROADCAST NONE)',
 'PQ_DISTRIBUTE(@"MRG$1" "RMS"@"MRG$1" NONE BROADCAST)',
 'USE_MERGE_CARTESIAN(@"MRG$1" "H"@"MRG$1")',
 'USE_MERGE_CARTESIAN(@"MRG$1" "RMS"@"MRG$1")',
 'LEADING(@"MRG$1" "from$_subquery$_010"@"MRG$1" "RMS"@"MRG$1" "H"@"MRG$1")',
 'NO_ACCESS(@"MRG$1" "H"@"MRG$1")',
 'FULL(@"MRG$1" "RMS"@"MRG$1")',
 'NO_ACCESS(@"MRG$1" "from$_subquery$_010"@"MRG$1")',
 'OUTLINE(@"SEL$2")',
 'OUTLINE(@"SEL$1")',
 'OUTLINE_LEAF(@"MRG$1")',
 'OUTLINE_LEAF(@"SEL$5")',
 'OUTLINE_LEAF(@"SEL$6")',
 'MERGE(@"SEL$2")',
 'OUTLINE_LEAF(@"SEL$F5BB74E1")',
 'OUTLINE_LEAF(@"SEL$3")',
 'OUTLINE_LEAF(@"SEL$4")',
 'OPT_PARAM(''optimizer_index_caching'' 80)',
 'OPT_PARAM(''optimizer_index_cost_adj'' 3)',
 'OPT_PARAM(''_optimizer_connect_by_cost_based'' ''false'')',
 'OPT_PARAM(''_optim_peek_user_binds'' ''false'')',
 'OPT_PARAM(''_b_tree_bitmap_plans'' ''false'')',
 'DB_VERSION(''11.1.0.7'')',
 'OPTIMIZER_FEATURES_ENABLE(''11.1.0.7'')',
 'IGNORE_OPTIM_EMBEDDED_HINTS',
 'END_OUTLINE_DATA'
);
for sql_rec in (
 select t.sql_id, t.sql_text
 from dba_hist_sqltext t, dba_hist_sql_plan p
 where t.sql_id = p.sql_id and p.sql_id = 'dgh672sad89j4' and p.plan_hash_value = 12345678 and p.parent_id is null
 ) loop
 DBMS_SQLTUNE.IMPORT_SQL_PROFILE(
 sql_text => sql_rec.sql_text,profile => ar_profile_hints,name => 'PROFILE_'||sql_rec.sql_id
 );
 end loop;
end;
/

That was simple. Isn’t it??

But i was not satisfied. I wanted to take this a step further. Now i want oracle to use whatever plan I want.

In case oracle had changed plan , I want oracle to use previous plan. 

In one of my database, planflip caused cpu spikes. Buffer_gets/exec increased from thousands to millions. The idea was to force old to use previous plan .

oldprofile.sql

</pre>
Sesion1>@oldprofile
enter sql_id:- 9ddv4vbt2zrxy
enter old plan hash value:- 1236769290
enter bad plan hash value to be tuned :- 3891475932
'===================OUTPUT========================='

declare
ar_profile_hints sys.sqlprof_attr;
begin
 ar_profile_hints := sys.sqlprof_attr(
 'BEGIN_OUTLINE_DATA',
 'IGNORE_OPTIM_EMBEDDED_HINTS',
 'OPTIMIZER_FEATURES_ENABLE(''11.1.0.7'')',
 'DB_VERSION(''11.1.0.7'')',
 'OPT_PARAM(''_b_tree_bitmap_plans'' ''false'')',
 'OPT_PARAM(''_optim_peek_user_binds'' ''false'')',
 'OPT_PARAM(''_optimizer_connect_by_cost_based'' ''false'')',
 'OPT_PARAM(''optimizer_index_cost_adj'' 3)',
 'OPT_PARAM(''optimizer_index_caching'' 80)',
 'OUTLINE_LEAF(@"SEL$1")',
 'INDEX(@"SEL$1" "CCS"@"SEL$1" ("COMPLETED_CUSTOMER_SHIPMENTS"."CONDITION" "COMPLETED_CUSTOMER_SHIPMENTS"."LAST_UPDATED"))',
 'END_OUTLINE_DATA'
);
for sql_rec in (
 select t.sql_id, t.sql_text
 from dba_hist_sqltext t, dba_hist_sql_plan p
 where t.sql_id = p.sql_id and p.sql_id = '9ddv4vbt2zrxy' and p.plan_hash_value = 3891475932 and p.parent_id is null
 ) loop
 DBMS_SQLTUNE.IMPORT_SQL_PROFILE(
 sql_text => sql_rec.sql_text,profile => ar_profile_hints,name => 'PROFILE_'||sql_rec.sql_id
 );
 end loop;
end;
/

Ever encountered issue when your query does not pick baseline. In some cases, even after setting fixed=Yes, it does not pick baseline.

This may happen when you try fixing the query using hints and try to link the plan of hinted query with original query.

This post is intended to provide you the solutions for some of those issues. (And like every post a script for the dba’s.)

And most important, do we need profiles now.

Why oracle is not picking my baseline

There are several reasons(and bugs) why oracle  does not pick baselines.

In my case, I tried to manually tune the query using hints and linked the plan with the original query. However, oracle refused to pick the baseline. 

In many cases reason could be

1. Bind Variables used in the query
2. Aliases used  for the table

Solution 1:-

Using bind variables prevents oracle from generating multiple versions of the same query and helps in reducing hard parsing. But what if the same bind variables creates trouble for you.

If  your application is using bind variable or cursor_sharing is set to “similar”or “force”, then your query should look like this.

For the test case, I will use mytab table

Session1> select  * from mytab a where a.generated=:b1;

no rows selected

You can always check the values of bind variables using following query.

Session2>select name,value_string from v$sql_bind_capture where sql_id='&sql_id' and child_number='&child';

Enter value for sql_id: 3uj38fuby1shm
Enter value for child: 0
NAME                 VALUE_STRING
-------------------- ------------------------------
:B1                  FOR
1 row selected.

In cases, where query is using bind variables like the one given above, use following approach to tune the query and the link the baseline

var b1 varchar2(20);
exec :b1:='FOR';
Session1 >select /*+ index(a MYTAB_IDX2) */ * from mytab a where a.generated=:b1;
no rows selected

Now, we can link the plan of the tuned sql with the original sql as described in earlier post.

Solution 2:-

You may need to tune the sql’s running inside plsql packages and procedures. In those cases, bind variables name could be like “:1″ which is hard to tune using traditional approach of baselines.

e.g. select * from mytab a where a.generated=:1;

To tune those sql’s you need to create a anonymous block and create a cursor like one given below.

declare
sql_stmt varchar2(4000);
id varchar2(20);
TYPE CurTyp  IS REF CURSOR;
tmpcursor    CurTyp;
begin
   id:='&id_val';
  sql_stmt:='select /*+ index(a MYTAB_IDX2) */ * from mytab where generated=:1';
  open tmpcursor for sql_stmt using id;
end;
/
Enter value for id_val: FOR

PL/SQL procedure successfully completed.

Now, you can find the sql_id of the query and link the baseline with original sql.

Session1>select sql_id,module,sql_text from v$sql where upper(sql_text) like upper('%&text%') and ( sql_text not like lower('%v$sql%') and sql_text not like '%explain plan%') group by sql_id,module,sql_text;
Enter value for text: select /*+ index(a MYTAB_IDX2) */ * from mytab where generated=:1

SQL_ID        MODULE                    SQL_TEXT
------------- ------------------------- ------------------------------------------------------------------------------------------
5vkq50nxn4gsj SQL*Plus                  declare sql_stmt varchar2(4000); id varchar2(20); TYPE CurTyp  IS REF CURSOR; tmpcursor
                                         CurTyp; begin    id:='FOR';   sql_stmt:='select /*+ index(a MYTAB_IDX2) */ * from mytab w
                                        here generated=:1';   open tmpcursor for sql_stmt using id; end;

9z3dv649q3gvy SQL*Plus                  select /*+ index(a MYTAB_IDX2) */ * from mytab where generated=:1

2 rows selected.

You can link the plan of 9z3dv649q3gvy with original query to tune it.

Solution 3:-

The second solution of using cursor may work well if query is small or using limited number of bind variables. But for page long queries, its cumbersome to concatenate multiple bind variables.

We can use profiles to bypass second approach. (profile.sql)

Profile.sql will ask for sql_id and child number of hinted sql and the sql_id and plan hash value of the bad sql_id. You can paste the output directly to create a profile.

Session1>@profile
enter hinted sql_id:- 9z3dv649q3gvy
enter child number:- 0
enter original sql_id:- 9jutgyhjutefd
enter bad plan hash value to be tuned :- 663241269
'===================OUTPUT========================='

declare
ar_profile_hints sys.sqlprof_attr;
begin
        ar_profile_hints := sys.sqlprof_attr(
                'BEGIN_OUTLINE_DATA',
                'IGNORE_OPTIM_EMBEDDED_HINTS',
                'OPTIMIZER_FEATURES_ENABLE(''11.1.0.7'')',
                'DB_VERSION(''11.1.0.7'')',
                'OPT_PARAM(''_b_tree_bitmap_plans'' ''false'')',
                'OPT_PARAM(''_optim_peek_user_binds'' ''false'')',
                'OPT_PARAM(''_optimizer_connect_by_cost_based'' ''false'')',
                'OPT_PARAM(''optimizer_index_cost_adj'' 1)',
                'OPT_PARAM(''optimizer_index_caching'' 80)',
                'OUTLINE_LEAF(@"SEL$1")',
                'INDEX_RS_ASC(@"SEL$1" "MYTAB"@"SEL$1" ("MYTAB"."GENERATED"))',
                'END_OUTLINE_DATA'
);
for sql_rec in (
        select t.sql_id, t.sql_text
         from dba_hist_sqltext t, dba_hist_sql_plan p
        where t.sql_id = p.sql_id and p.sql_id = '9jutgyhjutefd' and p.plan_hash_value = 663241269 and p.parent_id is null
                ) loop
              DBMS_SQLTUNE.IMPORT_SQL_PROFILE(
                sql_text    => sql_rec.sql_text,profile     => ar_profile_hints,name => 'PROFILE_'||sql_rec.sql_id
                                        );
        end loop;
end;
/

You can paste the output directly to fix the sql.

In my previous post,  I had described how to implement hints through baselines without changing code. Also, I touched upon two parameters namely

1. Optimizer_capture_sql_plan_baselines              2. Optimizer_use_sql_plan_baseline 

As I had written “optimizer_capture_sql_plan_baseline if set to true, let oracle automatically capture sql plan baselines.”

Does it mean it will not capture any plan for any sql if set to false ?

What if we want to capture baselines for only selected queries automatically.

The whole answer lies in values of “ENABLED,ACCEPTED AND FIXED” column if dba_sql_plan_baselines. How!!!!!? Lets find out…

As usual, lets create a table and insert few records

Session1>show parameter capture
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
optimizer_capture_sql_plan_baselines boolean     FALSE

create table test (name varchar2(10), id number(*));

begin
for i in 1..1000
loop
insert into test values ('SUMIT',i);
end loop;
commit;
end;
/
begin
for i in 1001..10000
loop
insert into test values ('BHATIA',i);
end loop;
commit;
end;
/
begin
for i in 10001..100000
loop
insert into test values ('BASELINE',i);
end loop;
commit;
end;
/
begin
for i in 100001..1000000
loop
insert into test values ('SPM',i);
end loop;
commit;
end;
/

SELECT COUNT(*),name from test group by name order by 1;

COUNT(*) NAME
---------- ----------
1000 SUMIT
9000 BHATIA
90000 BASELINE
900000 SPM

4 rows selected.

Time to query table and load the plan into the baseline.

--Inserting hint to easily identify the query
Select /*+ id=1000 */  * FROM TEST where id=1000
NAME               ID
---------- ----------
SUMIT            1000

select sql_id,exact_matching_signature,force_matching_signature from v$sql where sql_text like '%id=1000%';
SQL_ID                EXACT_MATCHING_SIGNATURE         FORCE_MATCHING_SIGNATURE
------------- -------------------------------- --------------------------------
duk2ypk5fz9g6              7784548270786280511              4428329137525653294

SELECT * FROM TABLE(dbms_xplan.display_cursor('duk2ypk5fz9g6'));
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------
SQL_ID  duk2ypk5fz9g6, child number 0
-------------------------------------
select /*+ id=1000 */  * FROM TEST where id=1000
Plan hash value: 1357081020
----------------------------------
| Id  | Operation         | Name |
----------------------------------
|   0 | SELECT STATEMENT  |      |
|*  1 |  TABLE ACCESS FULL| TEST |
----------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("ID"=1000)
Note
-----
- rule based optimizer used (consider using cbo)

--Creating the baseline for the plan

var v_num number;
exec :v_num:=dbms_spm.load_plans_from_cursor_cache(sql_id => 'duk2ypk5fz9g6',plan_hash_value => 1357081020 );

select sql_handle, plan_name, enabled, accepted,fixed from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX
------------------------------ ------------------------------ --- --- ---
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO

As we can see, If we load the plan manually, that will plan will be enabled and accepted but not fixed.  Its time create an index and run the query again.

create index test_id on test(id);
Index created.

select /*+ id=1000 */  * FROM TEST where id=1000
NAME               ID
---------- ----------
SUMIT            1000

Oracle is still using SYS_SQL_PLAN_7118fc3f97bbe3d0 baseline.

However, if we query dba_sql_plan_baselines, we will see that oracle has started capturing baselines for this sql. (plan name SYS_SQL_PLAN_7118fc3f642e4a26)

PLAN_TABLE_OUTPUT
-----------------------------------------------------------
SQL_ID  duk2ypk5fz9g6, child number 1
-------------------------------------
select /*+ id=1000 */  * FROM TEST where id=1000
Plan hash value: 1357081020
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |       |       |   554 (100)|          |
|*  1 |  TABLE ACCESS FULL| TEST |  1643 | 32860 |   554   (2)| 00:00:07 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("ID"=1000)
Note
-----
- SQL plan baseline SYS_SQL_PLAN_7118fc3f97bbe3d0 used for this statement

select sql_handle, plan_name, enabled, accepted,fixed,signature from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX             SIGNATURE
------------------------------ ------------------------------ --- --- --- ---------------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES NO NO   7784548270786280511
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO   7784548270786280511

So even if capture baseline is set to false, oracle will still capture the new plans and baselines for that particular sql onwards. We can evolve the  new plan and can check if index is getting used or not.

/*Evolving new plan */
SET SERVEROUTPUT ON
 SET LONG 10000
 set lines 175
 DECLARE
 report clob;
 BEGIN
 report := DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE(
 sql_handle => '&sql_handle', time_limit => &time);
 DBMS_OUTPUT.PUT_LINE(report);
 END;
 /
 Enter value for sql_handle:SYS_SQL_6c0845687118fc3f
Enter value for time:60

-------------------------------------------------------------------------------
 Evolve SQL Plan Baseline
 Report
 -------------------------------------------------------------------------------
Inputs:
 -------
 SQL_HANDLE = SYS_SQL_6c0845687118fc3f
 PLAN_NAME  =
 TIME_LIMIT = 60
VERIFY     = YES
 COMMIT     = YES
Plan: SYS_SQL_PLAN_7118fc3f642e4a26
 -----------------------------------
 Plan was verified: Time used .05 seconds.
 Passed
 performance criterion: Compound improvement ratio >= 504.88
 Plan was changed to an accepted plan.
Baseline Plan      Test Plan     Improv. Ratio
-------------      ---------     -------------
 Execution Status:        COMPLETE       COMPLETE
 Rows Processed:                 1              1
 Elapsed Time(ms):               26             0
 CPU Time(ms):                  25              0
 Buffer Gets:                 2017              4            504.25
 Disk Reads:                     0              0
 Direct Writes:                  0              0
 Fetches:                        0              0
 Executions:                     1              1
-------------------------------------------------------------------------------
 Report
 Summary
 -------------------------------------------------------------------------------
 Number of SQL plan baselines verified: 1.
 Number of SQL plan baselines evolved: 1.

As we can read in the report, Oracle has marked the new plan as accepted. Oracle keeps a tab if the plan was loaded manually or captured automatically. This can be verified from origin column of dba_sql_plan_baselines.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES YES NO AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO MANUAL-LOAD

So far, the story was pretty much straight. Here’s a quick recap what we have done
1. Created Table and Inserted records
2. Query the table and baseline the query
3. Created Index and verified new plan is loaded but not accepted.
4. Evolve the new plan and accepted=yes.
5. New plan(Index Range Scan) started getting used.

Its time to discuss the usage of enabled,accepted and fixed parameters.

TEST1:  ENABLED=NO.

For every sql_id, oracle checks if the baselines exists or not. If yes, then it will filter all the enabled plans. The plans for which enabled=NO will not be considered even if accepted and fixed are set to yes.

Other than this, if any plan is set enabled=yes for a query,oracle will start capturing new plans also.

Therefore, enabled is the first level. If enabled is set to NO,oracle will mark not to use and evolve that plan.

Lets have a test case. I will mark enabled =no for the plan that was using index and check which plan is used.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES YES NO AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO MANUAL-LOAD

/* Setting Enabled=NO for SYS_SQL_PLAN_7118fc3f642e4a26 */
declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle => '&sql_handle',plan_name  => '&plan_name',attribute_name => 'ENABLED',   attribute_value => 'NO');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f642e4a26

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  NO YES  NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO  MANUAL-LOAD
2 rows selected.

So, plan SYS_SQL_PLAN_7118fc3f642e4a26 is accepted but not enabled. Lets execute the query again and see which plan is getting used.

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------
SQL_ID  duk2ypk5fz9g6, child number 1
-------------------------------------
select /*+ id=1000 */  * FROM TEST where id=1000
Plan hash value: 1357081020
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |       |       |   554 (100)|          |
|*  1 |  TABLE ACCESS FULL| TEST |  1643 | 32860 |   554   (2)| 00:00:07 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("ID"=1000)
Note
-----
- SQL plan baseline SYS_SQL_PLAN_7118fc3f97bbe3d0 used for this statement

Oracle choses SYS_SQL_PLAN_7118fc3f97bbe3, since that plan is still enabled and accepted.
Therefore, minimum requirement for oracle to use a plan as baseline is, it should be marked both as ENABLED and ACCEPTED.

The next level is Accepted. In this level oracle will pick all the plans that are accepted to the users and discard all “ACCEPTED=NO” plans. As we had already seen,there can be more than one plan for accepted=yes. All these plans are already evolved plans and are candidate for being  the final execution plan.

If no plan is marked “FIXED=YES”, then oracle will pick any of these plans based on the costing.

All those plans which are “ENABLED=YES” and “ACCEPTED=NO”, are candidates for evolution. We can also however marked then accepted=yes manually.

Next and the last level is Fixed. If any plan is fixed oracle will use that plan only. If more than one plan is fixed, oracle will use costing as criteria to select the plan among those.

TEST2:  Setting any one plan as FIXED=YES

Lets mark index plan as accepted again and FTS plan as fixed.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  NO  YES NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO  MANUAL-LOAD

/*Resetting SYS_SQL_PLAN_7118fc3f642e4a26 to Enabled */
declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle =>'&sql_handle',plan_name=>'&plan_name',attribute_name =>'ENABLED',attribute_value =>'YES');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f642e4a26

/*Setting SYS_SQL_PLAN_7118fc3f97bbe3d0 to FIXED */
declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle =>'&sql_handle',plan_name=>'&plan_name',attribute_name=>'FIXED',attribute_value=>'YES');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f97bbe3d0

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES YES NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD

Executing the query and checking the plan again

SELECT * FROM TABLE(dbms_xplan.display_cursor('duk2ypk5fz9g6'));
PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------
SQL_ID  duk2ypk5fz9g6, child number 1
-------------------------------------
select /*+ id=1000 */  * FROM TEST where id=1000
Plan hash value: 1357081020
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |       |       |   554 (100)|          |
|*  1 |  TABLE ACCESS FULL| TEST |  1643 | 32860 |   554   (2)| 00:00:07 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("ID"=1000)
Note
-----
- SQL plan baseline SYS_SQL_PLAN_7118fc3f97bbe3d0 used for this statement

As we can confirm oracle start using FTS as that plan is marked fixed.

TEST3: Setting FIXED=YES for more than one plan

Lets mark second plan “FIXED=YES” and re-execute the query

/*Setting SYS_SQL_PLAN_7118fc3f642e4a26 to FIXED */
declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle=>'&sql_handle',plan_name=>'&plan_name',attribute_name=>'FIXED',attribute_value=>'YES');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f642e4a26

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;

SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES YES YES AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD

select /*+ id=1000 */  * FROM TEST where id=1000
NAME               ID
---------- ----------
SUMIT            1000

1 row selected.

SELECT * FROM TABLE(dbms_xplan.display_cursor('duk2ypk5fz9g6'));

PLAN_TABLE_OUTPUT
---------------------------------------------------------------
SQL_ID  duk2ypk5fz9g6, child number 0
-------------------------------------
select /*+ id=1000 */  * FROM TEST where id=1000
Plan hash value: 1699862855
---------------------------------------------------------------------------------------
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |       |       |     1 (100)|          |
|   1 |  TABLE ACCESS BY INDEX ROWID| TEST    |     1 |    20 |     1   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | TEST_ID |     1 |       |     1   (0)| 00:00:01 |
---------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("ID"=1000)
Note
-----
- dynamic sampling used for this statement
- SQL plan baseline SYS_SQL_PLAN_7118fc3f642e4a26 used for this statement

Test 4:

Also, if any of the plan is marked as fixed, oracle will stop capturing and evolving new plans.
To simulate the test, lets again mark the plan(using index) as fixed=no and accepted=no.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;

SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES YES YES AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD

/* Resetting SYS_SQL_PLAN_7118fc3f642e4a26 to FIXED=NO */
declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle => '&sql_handle',plan_name  => '&plan_name',attribute_name => 'FIXED',   attribute_value => 'NO');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f642e4a26

/* Resetting SYS_SQL_PLAN_7118fc3f642e4a26 to ACCEPTED=NO */
declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle => '&sql_handle',plan_name  => '&plan_name',attribute_name => 'ACCEPTED',   attribute_value => 'NO');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f642e4a26

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES NO  NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD

Lets try to evolve the index plan (SYS_SQL_PLAN_7118fc3f642e4a26) again, like we did previously. The only difference is fixed=yes this time for plan using FTS

/* TRYING TO EVOLVE SYS_SQL_PLAN_7118fc3f642e4a26 */
SET SERVEROUTPUT ON
SET LONG 10000
set lines 175
DECLARE
report clob;
BEGIN
report := DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE(
sql_handle => '&sql_handle', time_limit => &time);
DBMS_OUTPUT.PUT_LINE(report);
END;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for time: 60
-------------------------------------------------------------------------------
Evolve SQL Plan Baseline
Report
-------------------------------------------------------------------------------
Inputs:
-------
SQL_HANDLE = SYS_SQL_6c0845687118fc3f
PLAN_NAME  =
TIME_LIMIT = 60
VERIFY     = YES
COMMIT     = YES
-------------------------------------------------------------------------------
Report
Summary
-------------------------------------------------------------------------------
There were no SQL plan baselines that required processing.

As we can confirm, plan SYS_SQL_PLAN_7118fc3f642e4a26 is not evolved since SYS_SQL_PLAN_7118fc3f97bbe3d0 was already fixed.

TEST5:

Lets go back a step further and drop the baseline as well as index. We will try to simulate a test case to check if oracle captures new plan if any plan is marked as FIXED=YES

SET SERVEROUTPUT ON
DECLARE
l_plans_dropped  PLS_INTEGER;
BEGIN
l_plans_dropped := DBMS_SPM.drop_sql_plan_baseline (
sql_handle => 'SYS_SQL_6c0845687118fc3f',
plan_name  => 'SYS_SQL_PLAN_7118fc3f642e4a26');

DBMS_OUTPUT.put_line(l_plans_dropped);
END;
/

drop index test_id;
Index dropped.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD
1 row selected.

So, we are back to square one from where we have started. Lets again create the same index and query the table to see if it still capture new plans.

create index test_id on test(id);
Index created.

select /*+ id=1000 */  * FROM TEST where id=1000;
NAME               ID
---------- ----------
SUMIT            1000
1 row selected.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;

SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD

1 row selected.

As we can see, oracle has stopped capturing new plans for this sql as SYS_SQL_PLAN_7118fc3f97bbe3d0 is already fixed.

Lets mark fixed=NO again and re-execute the query. Oracle should capturing the plan again for this query.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES YES MANUAL-LOAD
1 row selected.

declare
myplan pls_integer;
begin
myplan:=DBMS_SPM.ALTER_SQL_PLAN_BASELINE (sql_handle => '&sql_handle',plan_name  => '&plan_name',attribute_name => 'FIXED',   attribute_value => 'NO');
end;
/
Enter value for sql_handle: SYS_SQL_6c0845687118fc3f
Enter value for plan_name: SYS_SQL_PLAN_7118fc3f97bbe3d0

select /*+ id=1000 */  * FROM TEST where id=1000
NAME               ID
---------- ----------
SUMIT            1000
1 row selected.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;

SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES NO  NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES YES NO  MANUAL-LOAD

2 rows selected.

So, as we can confirm oracle again started capturing new plans we turn FIXED=NO.

Here’s the quick summary,

1.  ENABLED=YES (For any of the plan) :- Oracle will start capturing new plans for those queries.

2. Enabled=NO :-  Than plan won’t we used.

3.  Accepted=Yes (Any one Plan) :- That plan will be used for execution

4.  Accepted=YES (For Multiple plans):- Any one plan can be used if fixed=no for all plans.

5. Accepted=No: Plan wont be used.

6.  FIXED=YES (for only one plan) Only that plan will be used and oracle will stop capturing/evolving new plans.

7.  FIXED=YES (for many plans) Oracle will chose execution plan only from that pool.

SPECIAL CASE :-ENABLED=YES,FIXED=YES BUT ACCEPTED=NO.

What if in our given scenario, we mark SYS_SQL_PLAN_7118fc3f97bbe3d0 as fixed and enabled but not accepted. Will oracle capture the new plan and will it evolve it.

select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES NO  YES MANUAL-LOAD
1 row selected.

Lets discuss the above scenario and try to figure out who takes precedence under what condition

select /*+ id=1000 */  * FROM TEST where id=1000;
NAME               ID
---------- ----------
SUMIT            1000
1 row selected.
select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;
SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES NO  NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES NO  YES MANUAL-LOAD
2 rows selected.

Oracle has captured new plan. Lets try to evolve that one also.

SET SERVEROUTPUT ON
SET LONG 10000
set lines 175
DECLARE
report clob;
BEGIN
report := DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE(
sql_handle => '&sql_handle', time_limit => &time);
DBMS_OUTPUT.PUT_LINE(report);
END;
/
-------------------------------------------------------------------------------
Evolve SQL Plan Baseline
Report
-------------------------------------------------------------------------------

Inputs:
-------
SQL_HANDLE = SYS_SQL_6c0845687118fc3f
PLAN_NAME  =
TIME_LIMIT =
60
VERIFY     = YES
COMMIT     = YES

-------------------------------------------------------------------------------
Report
Summary
-------------------------------------------------------------------------------
There were no SQL plan baselines that required processing.
select sql_handle, plan_name, enabled, accepted,fixed,origin from dba_sql_plan_baselines where signature=7784548270786280511;

SQL_HANDLE                     PLAN_NAME                      ENA ACC FIX ORIGIN
------------------------------ ------------------------------ --- --- --- --------------
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f642e4a26  YES NO  NO  AUTO-CAPTURE
SYS_SQL_6c0845687118fc3f       SYS_SQL_PLAN_7118fc3f97bbe3d0  YES NO  YES MANUAL-LOAD

2 rows selected.

ENABLED takes precedence over FIXED in case of capturing the new plan and vice versa in case of evolving.

I hope above article will help in understanding the role/usage of ENABLE/ACCEPTED/FIXED  parameters. Please do write in case you find any discrepancy or want to share some details

SQL Plan Management(SPM) provides a mechanism to prevent unwanted plan flips.

Whenever a query is hard parsed, oracle produces several execution plan and compares it with the accepted plan in a baseline. If a match is found, oracle picks that plan.

The behaviour of baseline is governed by 2 parameters.

1. Optimizer_capture_sql_plan_baselines
2. Optimizer_use_sql_plan_baseline

optimizer_capture_sql_plan_baseline if set to true, let oracle automatically capture sql plan baselines. The default value is false

Optimizer_use_sql_plan_baseline, if set to true, force oracle to use the sql plan baseline (if present). The default is true.

Implementing HINTS without changing code

Environment Setting :-

• Optimizer_capture_sql_plan_baselines (false)
• Optimizer_use_sql_plan_baseline (true)

Session1>desc mytab
Name Null? Type
----------------------------------------- -------- ----------------------------
OWNER VARCHAR2(30)
OBJECT_NAME VARCHAR2(128)
SUBOBJECT_NAME VARCHAR2(30)
OBJECT_ID NUMBER
DATA_OBJECT_ID NUMBER
OBJECT_TYPE VARCHAR2(19)
CREATED DATE
LAST_DDL_TIME DATE
TIMESTAMP VARCHAR2(19)
STATUS VARCHAR2(7)
TEMPORARY VARCHAR2(1)
GENERATED VARCHAR2(1)
SECONDARY VARCHAR2(1)
NAMESPACE NUMBER
EDITION_NAME VARCHAR2(30)

select /*+woindex */ * from mytab where GENERATED='FOR';

no rows selected

SELECT * FROM TABLE(dbms_xplan.display_cursor('0ytzpgnhg1g83'));
SQL_ID  0ytzpgnhg1g83, child number 0
-------------------------------------
select /*+woindex */ * from mytab where GENERATED='FOR'
Plan hash value: 96696846
---------------------------------------------------------------------------
| Id  | Operation         | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |       |       |       |   124K(100)|          |
|*  1 |  TABLE ACCESS FULL| MYTAB |  4819K|   422M|   124K  (3)| 00:24:54 |
---------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("GENERATED"='FOR')

select sql_text, sql_handle, plan_name, enabled, accepted from dba_sql_plan_baselines;
no rows selected

var v_num number;
exec :v_num:=dbms_spm.load_plans_from_cursor_cache(sql_id => '0ytzpgnhg1g83',plan_hash_value => 96696846 );
PL/SQL procedure successfully completed.
select sql_text, sql_handle, plan_name, enabled, accepted from dba_sql_plan_baselines;
SQL_TEXT                                                                         SQL_HANDLE                     PLAN_NAME                      ENA ACC

-------------------------------------------------------------------------------- ------------------------------ ------------------------------ --- ---

select /*+woindex */ * from mytab where GENERATED='FOR'                          SYS_SQL_50969e88fdd635aa       SYS_SQL_PLAN_fdd635aada00620d  YES YES

1 row selected.

create index mytab_idx2 on mytab(GENERATED) parallel 4;
Index created.

alter index mytab_idx2 noparallel;

Index altered.

select * FROM TABLE(dbms_xplan.display_cursor('0ytzpgnhg1g83',1));
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  0ytzpgnhg1g83, child number 1
-------------------------------------
select /*+woindex */ * from mytab where GENERATED='FOR'
Plan hash value: 96696846
---------------------------------------------------------------------------
| Id  | Operation         | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |       |       |       |   124K(100)|          |
|*  1 |  TABLE ACCESS FULL| MYTAB |  4819K|   422M|   124K  (3)| 00:24:54 |
---------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter("GENERATED"='FOR')
Note
-----
   - SQL plan baseline SYS_SQL_PLAN_fdd635aada00620d used for this statement
22 rows selected.

select /*+index(mytab_idx2 mytab) */ * from mytab where GENERATED='FOR';

no rows selected

Select * FROM TABLE(dbms_xplan.display_cursor('ay757nb7anm56'));
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------
SQL_ID  ay757nb7anm56, child number 0
-------------------------------------
select /*+index(mytab_idx2 mytab) */ * from mytab where GENERATED='FOR'
Plan hash value: 3007699452
------------------------------------------------------------------------------------------
| Id  | Operation                   | Name       | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |            |       |       |   995 (100)|          |
|   1 |  TABLE ACCESS BY INDEX ROWID| MYTAB      |  4819K|   422M|   995   (1)| 00:00:12 |
|*  2 |   INDEX RANGE SCAN          | MYTAB_IDX2 |  4896K|       |    90   (2)| 00:00:02 |
------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("GENERATED"='FOR')
19 rows selected.

exec :v_num:=dbms_spm.load_plans_from_cursor_cache(sql_id => 'ay757nb7anm56',plan_hash_value => '3007699452' );
select sql_text, sql_handle, plan_name, enabled, accepted from dba_sql_plan_baselines;
SQL_TEXT                                                                         SQL_HANDLE                     PLAN_NAME                      ENA ACC

-------------------------------------------------------------------------------- ------------------------------ ------------------------------ --- ---
select /*+woindex */ * from mytab where GENERATED='FOR'                          SYS_SQL_50969e88fdd635aa       SYS_SQL_PLAN_fdd635aa041dae64  YES NO

select /*+woindex */ * from mytab where GENERATED='FOR'                          SYS_SQL_50969e88fdd635aa       SYS_SQL_PLAN_fdd635aada00620d  YES YES

select /*+index(mytab_idx2 mytab) */ * from mytab where GENERATED='FOR'          SYS_SQL_5229297eca7bb2c7       SYS_SQL_PLAN_ca7bb2c7041dae64  YES YES

3 rows selected.

exec :v_num:=dbms_spm.load_plans_from_cursor_cache(sql_id => 'ay757nb7anm56',plan_hash_value => 3007699452 ,sql_handle => 'SYS_SQL_50969e88fdd635aa');

select sql_text, sql_handle, plan_name, enabled, accepted from dba_sql_plan_baselines;
SQL_TEXT                                                                         SQL_HANDLE                     PLAN_NAME                      ENA ACC
-------------------------------------------------------------------------------- ------------------------------ ------------------------------ --- ---
select /*+woindex */ * from mytab where GENERATED='FOR'                          SYS_SQL_50969e88fdd635aa       SYS_SQL_PLAN_fdd635aa041dae64  YES YES

select /*+woindex */ * from mytab where GENERATED='FOR'                          SYS_SQL_50969e88fdd635aa       SYS_SQL_PLAN_fdd635aada00620d  YES YES

select /*+index(mytab_idx2 mytab) */ * from mytab where GENERATED='FOR'          SYS_SQL_5229297eca7bb2c7       SYS_SQL_PLAN_ca7bb2c7041dae64  YES YES

3 rows selected.

 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 OWNER                                              VARCHAR2(30)
 OBJECT_NAME                                        VARCHAR2(128)
 SUBOBJECT_NAME                                     VARCHAR2(30)
 OBJECT_ID                                          NUMBER
 DATA_OBJECT_ID                                     NUMBER
 OBJECT_TYPE                                        VARCHAR2(19)
 CREATED                                            DATE
 LAST_DDL_TIME                                      DATE
 TIMESTAMP                                          VARCHAR2(19)
 STATUS                                             VARCHAR2(7)
 TEMPORARY                                          VARCHAR2(1)
 GENERATED                                          VARCHAR2(1)
 SECONDARY                                          VARCHAR2(1)
 NAMESPACE                                          NUMBER
 EDITION_NAME                                       VARCHAR2(30)

Session1>@tableinfo
enter table name :- mytab

INDEX_NAME                     COLUMN_NAME                              COLUMN_POSITION
------------------------------ ---------------------------------------- ---------------
MYTAB_IDX                      OWNER                                                  1
MYTAB_IDX                      OBJECT_NAME                                            2
MYTAB_IDX                      SUBOBJECT_NAME                                         3
MYTAB_IDX2                     GENERATED                                              1

(Size) Mb       Size(Gb)
----------     ----------
3581.5      3.49755859

As we have seen in previous posts journal table behaviour, if the rebuild gets cancelled in between and oracle does not gets a latch, it does not clear the journal table automatically.

Also, oracle won’t allow dropping or rebuilding of index.

session1>alter index mytab_idx rebuild online;
alter index mytab_idx rebuild online
*
ERROR at line 1:
ORA-08104: this index object 98701 is being online built or rebuilt

sesion1>drop index mytab_idx;
drop index mytab_idx
           *
ERROR at line 1:
ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired

Its the responsibility of the smon to clear the indexes stuck in journal.

However, in many cases dba intervention is required.

There are 2 ways to clear the journal table.

1. By waking up smon

2. By using oracle supplied dbms_repair.online_index_clean package

waking up smon

We can post smon to clear the temporary segments and journal by issuing the following command.

select pid from v$process p, v$bgprocess b where b.paddr = p.addr and name='SMON';

oradebug wakeup <pid>

Using dbms_repair.online_index_clean package

dbms_repair is a oracle supplied package which has many procedures to detect and repair block corruption or rebuild freelists etc.

we can use online_index_clean to clean up the journal table.

Be informed that this procedure also requires initial latch. I have developed a small script ndx-cleanup.sql to clear the journal table. It takes owner and index name as input and must be executed from SYS.                                                                                                                                                                      

Session1>@ndx-cleanup
Enter value for owner: subhatia_dba
Enter value for index_name: mytab_idx
OBJECT ID OF INDEX IS -----------> 98701
JOURNAL TABLE IS -----------> SYS_JOURNAL_98701
CURRENT INDEX STATUS -----------> VALID
INDEX CLEANUP COMPLETED
JOURNAL TABLE HAS BEEN CLEANED

PL/SQL procedure successfully completed.

1. Behaviour of journal table.
2. Effects of killing index in between.

Please click here to understand the basic structure of journal table.

Behaviour of Journal Table

As discussed earlier, journal tables are created when we issue alter index command and gets dropped only after the job completes successfully or cleans or cancelled successfully.  Whenever we kill our session or cancel our job, oracle tries to clean the journal table . If you have killed your job (CTL C), oracle tries to get a latch and then merge the entries and then drop the journal table.

 In case it does not, it will try fixed number of times and will then fail. This behaviour is explained in detail in later section. However, we must verify that journal tables are no longer present.

Test Case:-

From session 1 insert a row  and issue rebuild comand from session 2 and then hit ctl c to cancel the job.

Session1>insert into mytab select * from mytab where rownum<2;

Session2>alter index mytab_idx rebuild online;
alter index mytab_idx rebuild online
*
ERROR at line 1:
ORA-01013: user requested cancel of current operation<

/*it will error out after 2000 retries or 20000 milli seconds. Discussed later */

Lets check if the journal table exists or not.

sesion2>select object_name,object_type,object_id from dba_objects where object_name='MYTAB_IDX';

OBJECT_NAME          OBJECT_TYPE           OBJECT_ID
-------------------- -------------------- ----------
MYTAB_IDX            INDEX                     98701

1 row selected.

session2>select object_name,object_type from dba_objects where object_name='SYS_JOURNAL_98701';

OBJECT_NAME          OBJECT_TYPE
-------------------- --------------------
SYS_JOURNAL_98701    TABLE

1 row selected.

As suspected , oracle timesout and left the cleaning part. All the transaction entries will now go into the journal table from now since the trigger has not been changed back to point to the index. Its still pointing to the journal table. As we can see,if the index rebuild job does not perform cleaning, the entries will stuck in journal table and we have to take manual steps to recover it.

session1>select * from SYS_JOURNAL_98701;

session1>insert into mytab(owner,object_name) values ('AFTER INDEX','CANCEL');

1 row created.
session1>commit;

Commit complete.

C0                   C1         C2    OPCODE     PARTNO RID
-------------------- ---------- ----- ---------- ------ ------------------
AFTER INDEX          CANCEL           I               0 D/////AAEAAAwQHAAL

1 row selected.

Effects of killing index in between

As we had already discussed that even after killing index job, oracle sometimes does not able to drop the journal table. Oracle will try exactly 2000 times with avg waitng time of 10 millli second (20,000 milli seconds in all), in case all the blocking sessions are inactive,  to acquire a latch to clean the journal table.

Test Case

Lets insert a row without commiting and start rebuilding from next session.

Session1>insert into mytab select * from mytab where rownum<2;

1 row created.

Session2>alter index mytab_idx rebuild online; 

Lets check the status from session 3 using our ndx-progress.sql

SESSION3>@ndx-progress
Enter value for sid: 2178

SID              ---->2178
SERIAL           ---->6537
MODULE           ---->SQL*Plus
MACHINE          ---->db-fc-admin-6001.iad6.amazon.com
USERNAME         ---->SUBHATIA_DBA
OSUSER           ---->subhatia
SQL TEXT         ---->alter index mytab_idx rebuild online
SQL ID           ---->7sk4sratkuvq8
JOURNAL NAME     ---->SYS_JOURNAL_98701
TABLE NAME       ---->MYTAB
EVENT_NAME       ---->enq: TX - row lock contention
IDLE SINCE       ---->223
CPU TIME         ---->35995
ELAPSED TIME     ---->219143299
JOURNAL COUNT    ---->0
STAGE            ---->Waiting for first latch.
LONGOPS          ---->INDEX REBUILD NOT STARTED YET
PROGRESS         ---->No Tx started after index rebuild has commited yet. There are 0 rows in journal table
------------SESSIONS BLOCKING FIRST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2154    9407    INACTIVE                249             0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com
------------SESSIONS THAT MAY BLOCK LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE

NO POTENTIAL BLOCKERS FOUND FOR THE LAST LATCH

As expected, our insert has blocked the index rebuild. Lets cancel the job(ctl c) and check whats happening now using ndx-progress.

Yes, ndx-progress can report that one too (amazing).

click here for ndx-progress

SESSION3>@ndx-progress
Enter value for sid: 2178
Index Rebuild has been cancelled. Cleaning going on
SID              ---->2178
SERIAL           ---->6537
MODULE           ---->SQL*Plus
MACHINE          ---->db-fc-admin-6001.iad6.amazon.com
USERNAME         ---->SUBHATIA_DBA
OSUSER           ---->subhatia
SQL TEXT         ---->alter index mytab_idx rebuild online
SQL ID           ---->7sk4sratkuvq8
JOURNAL NAME     ---->SYS_JOURNAL_98701
TABLE NAME       ---->MYTAB
EVENT_NAME       ---->index (re)build online cleanup
IDLE SINCE       ---->2116
CPU TIME         ---->2164672
ELAPSED TIME     ---->2113894321
JOURNAL COUNT    ---->0
------------SESSIONS BLOCKING CLEANING PROCESS-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2154    9407    INACTIVE                2145            0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com
------------CLEANUP PROCESS TIMOUT-------------
EVENT                                   TOTAL WAITS             TOTAL TIMEOUTS          TIME WAITED             AVG WAIT                MAX WAIT
index (re)build online cleanup          1967                    1967                    19799                   10.07                   10                  10

On checking the status, index job was cancelled but journal entry was still there

Session2>alter index mytab_idx rebuild online;
alter index mytab_idx rebuild online
*
ERROR at line 1:
ORA-01013: user requested cancel of current operation
SESSION3>select event,total_waits,total_timeouts,time_waited,average_wait,max_wait from v$session_event where sid=2178 and event='index (re)build online cleanup';
EVENT                          TOTAL_WAITS TOTAL_TIMEOUTS TIME_WAITED AVERAGE_WAIT   MAX_WAIT
------------------------------ ----------- -------------- ----------- ------------ ----------
index (re)build online cleanup        2000           2000       20131        10.07         10
1 row selected.

As we can check, oracle tries for 2000  times but timeout each time. it kept tring for 2000*10 ms and then error out.

1. Structure and understanding of journal table
2. Behaviour of journal table.
3. Effect of killing the index rebuild in between.
4. Clearing the journal table.

Structure of Journal Table 

 Oracle creates journal table inside the owner schema as soon as we issue “alter index rebuild online”. It doesn’t matter if oracle has acquired the first latch or not. The entries of all the transactions started after we issue the command will be logged into the journal table. Journal table is the index organized table and their name will essentially be SYS_JOURNAL_<index object_id>.

To understand index rebuilding check the following link.

Index Rebuild Online

The structure is as follows

  As you might have noticed, there is no “U” for update in opcode since update essentially is Delete+ Insert. Two entries will be recorded for each update statement in the journal table. 

Name                    Null?    Type
C0                      NOT NULL VARCHAR2(30) -->first column of index

C1                      NOT NULL VARCHAR2(128) -->second column of index(composite index)
C2                      NOT NULL VARCHAR2(30) -->third column of index (composite index)
OPCODE                           CHAR(1) -->I for insert, D for delete.
PARTNO                           NUMBER
RID                     NOT NULL ROWID -->rowid from 7th char to end

 C0,C1,C2(columns of index) and RID will form the primary key of the journal table and will have a unique index with the name SYS_IOT_TOP_<journal object id>

The columns of journal table(c0,c1,c2) will have same data type as of base table and will always be not null irrespective of constarints on base table.

TEST CASE

I have created a table mytab (select * from dba_objects) and created a composite index mytab_idx on (owner,object_name,subobject_name).

As I started the rebuild, i could see following entries in my schema.

session4> select object_name,object_type,object_id from dba_objects where owner='SUBHATIA_DBA';

OBJECT_NAME                                        OBJECT_TYPE          OBJECT_ID
-------------------------------------------------- ------------------- ----------
MYTAB                                              TABLE                    98699
SYS_JOURNAL_98701                                  TABLE                   101871
SYS_IOT_TOP_101871                                 INDEX                   101872
MYTAB_IDX                                          INDEX                    98701

4 rows selected.

As we can confirm the name of the journal table is SYS_JOURNAL_98701 (where 98701 is object id of index) and IOT index is SYS_IOT_TOP_101871  where 101871 is object_id of journal table.

Usage of Journal Tables

When we rebuild index online, oracle does not read from the existing index. Instead it reads the base table. So whats the use of existiing index.

Well, oracle put the existing index into read only mode and records all the new transaction entries into the journal table.

Journal table is like any other IOT table. So whenever, any select statement needs to traverse a row, it gets the rowid from the new journal table and then access the table using that rowid.

Question1) Does journal table is always there in the database

Answer) Journal tables are created as we issue the rebuild command and gets dropped as soon as the command finishes.

Question 2) Does journal table record entries of transactions which starts before the alter index rebuild command was issued.

Answer) No, it does not. It will only capture entries of those transactions that have started after we fired rebuild. Oracle doesn’t know you are going to fire rebuild command.

Question3) Does journal table records entries of transactions started after rebuild only after we commit.

Answer) Journal table is like any other table. In case you performed some DML while index rebuid is going on and does not commit, you will be able to see those entries into the journal table from same session but others won’t. Others will see only if you commit them. Oracle follows basic ACID principle for journal tables too.

Test case

Lets verify all the above points

Are journal tables there forever??

Currently I have only 2 objects in my schema. 

session4>select object_name,object_type,object_id from dba_objects where owner='SUBHATIA_DBA';

OBJECT_NAME                                        OBJECT_TYPE          OBJECT_ID
-------------------------------------------------- ------------------- ----------
MYTAB_IDX                                          INDEX                    98701
MYTAB                                              TABLE                    98699

2 rows selected.

Lets insert few rows from 1st session to block the rebuild and issue alter index command from 2nd session

SESSION1>insert into mytab select * from mytab where rownum<5;
4 rows created.
/*alter index from session 2 */
session2>alter index mytab_idx rebuild online;  -->will get stuck

/*lets check again from session 1 */

session1>select object_name,object_type,object_id from dba_objects where owner='SUBHATIA_DBA';

OBJECT_NAME                                        OBJECT_TYPE          OBJECT_ID
-------------------------------------------------- ------------------- ----------
MYTAB                                              TABLE                    98699
SYS_JOURNAL_98701                                  TABLE                   101877
SYS_IOT_TOP_101877                                 INDEX                   101878
MYTAB_IDX                                          INDEX                    98701

4 rows selected.

Does journal table record entries of transactions which starts before the alter index rebuild command was issued??

Check the count of journal table from the session holding the lock.

SESSION1>select count(*) from SYS_JOURNAL_98701;

  COUNT(*)
----------
         0

1 row selected.

Does journal table records entries of transactions started after rebuild only after we commit??

 Lets open a new session and insert few rows without committing.

session4>insert into mytab select * from mytab where rownum<5;

4 rows created.

session4>select count(*) from SYS_JOURNAL_98701;

  COUNT(*)
----------
         4

1 row selected.

/*lets query from some another session*/

SESSION1>select count(*) from SYS_JOURNAL_98701;

  COUNT(*)
----------
         0

1 row selected.

/*Lets commit and again check count from another session */

SESSION1>select count(*) from SYS_JOURNAL_98701;

  COUNT(*)
----------
         4

1 row selected.

Understanding of Journal Table

When we say, journal table hold entries then the next obvious question is what entries. How does it store and what does it mean.Moreover, is there any mechanism to understand those.  

The answer is YES, we can.

Lets try out this test case.

Insert a row from session1 to block the index rebuild and start rebuilding index from another session.  

SESSION1>insert into mytab select * from mytab where rownum<5;

4 rows created.
/* Start rebuild from Session 2. Your session will stuck here even if its a pretty small index*/
session2>alter index mytab_idx rebuild Online;
/*Open a new session and insert a row */
session3>insert into mytab(owner,object_name,subobject_name) values('SUMIT','RANDOM','RANDOM');
1 row created.
session3>SELECT * FROM SYS_JOURNAL_98701;

C0                             C1                   C2                             O     PARTNO RID
------------------------------ -------------------- ------------------------------ - ---------- ------------------
SUMIT                          RANDOM               RANDOM                         I          0 D/////AAEAAAwQHAAU

 /*Let's update a row as well */
session3>update mytab set owner='DBA' where owner='SUMIT' AND object_name='RANDOM';

1 row updated.

session3>SELECT * FROM SYS_JOURNAL_98701;

C0         C1         C2         O     PARTNO RID
---------- ---------- ---------- - ---------- ------------------
DBA        RANDOM     RANDOM     I          0 D/////AAEAAAwQHAAU
SUMIT      RANDOM     RANDOM     D          0 D/////AAEAAAwQHAAU

2 rows selected.

Here we can see 2 entries. But there should be 3. Remember update=D+I.

Oracle knew we have updated the row which we just inserted and the entry was already there. So it removes the earlier entry and added 2 new entries for update only.

Now, the entry for Sumit is marked as “D” wich was earlier “I” and a new entry “DBA”  has been added as “I”.

Last question, why oracle has not stored initial 6 letters of rowid and can we trace it back.

Yes, we can trace it back. We can use following query (but it is a bit slower)

session3>select owner,object_name,SUBOBJECT_NAME from mytab where substr(rowid,7)='AAEAAAwQHAAU';

OWNER                OBJECT_NAME                                        SUBOBJECT_NAME
-------------------- -------------------------------------------------- ------------------------------
DBA                  RANDOM                                             RANDOM

1 row selected.

Oracle is not showing inital 6 letters because those always refer to object id.

We can use this method which is relatively faster.

session3>select substr(rowid,1,6) from  mytab  where rownum<2;

SUBSTR(ROWID,1,6)
------------------
AAAYGL

1 row selected.
session3>select owner,object_name,SUBOBJECT_NAME from mytab where rowid='AAAYGLAAEAAAwQHAAU';

OWNER                OBJECT_NAME                                        SUBOBJECT_NAME
-------------------- -------------------------------------------------- ------------------------------
DBA                  RANDOM                                             RANDOM

1 row selected.

One way to avoid all these is to rebuild indexes.However, many of us might have faced issues while rebuilding indexes. Sometimes, it causes performance issues (due to enq TM contention) or the command won’t come out.

Ever wonder, why the index rebuilding is suddenly taking such a long time. Why my session is not coming out or exactly where is it stuck.

This post is intended to explore the internals of online index rebuild in oracle11g. I have developed a customized script which will tell exactly where the session is stuck.

http://fordba.wordpress.com/2011/03/30/ndx-progress-sql/

The script is just an anonymous plsql block and does not perform any DDL/DML.

It queries various v$ views to present the current picture before you and does not cause or have any adverse impact

For online index rebuild oracle requires 2 latches. One at the beginning and one at the end of the operation.

Online Index Rebuild Stages (No Parallel)

During Online index rebuild oracle performs following in the given order :-

1. Create the journal table (inside index owner schema) SYS_JOURNAL_<objectid_index>
2. Wait for the first latch (enq: TX – row lock contention)
3. As soon as it gets latch, will perform db file sequential read to merge journal entries
4. Will start scattered read of the base table
5. Will again perform the sequential read to merge the journal entries which got generated during scattered read.
6. Will perform direct path read temp.
7. wait for the end latch

The good thing about index rebuild in 11g is , it does not cause locking for the DML’s (enq TM contention, which was there in 10g).

Test Case

Environment:- Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 – 64bit Production

1. Create a new table and insert few records.

SESSION1>CREATE TABLE mytab as select * from dba_objects;
Table created.

SESSION1>insert into mytab select * from mytab;
12459 rows created.

SESSION1>/
12459 rows created.
SESSION1>create index mytab_idx on myt(owner,OBJECT_NAME,SUBOBJECT_NAME);

Index created.

You may want to insert more rows to grow the size of index.

2. Insert a new records into the table.

SESSION1>insert into mytab select * from mytab where rownum<10;
9 rows created.

3. Open a new session (note down its sid) and issue alter index command.

SESSION2>select sid from v$mystat where rownum<2;
SID
----------
2195
1 row selected.
SESSION2>ALTER INDEX MYTAB_IDX REBUILD ONLINE;

4. Open a new session to see exactly whats happening.

SESSION3>@ndx-progress.sql
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->enq: TX - row lock contention
JOURNAL COUNT    ---->0
STAGE            ---->Waiting for first latch.
PROGRESS         ---->No Tx started after index rebuild has commited yet. There are 0 rows in journal table
------------SESSIONS BLOCKING FIRST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2161    19807   INACTIVE                425             0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com
------------SESSIONS THAT MAY BLOCK LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
NO POTENTIAL BLOCKERS FOUND FOR THE LAST LATCH

5. In the above output, you can see the session is waiting on enq: TX – row lock contention and has not acquired the first latch.

It will keep on waiting for all the transactions which have started before issueing index rebuild.

For any transactions started after index rebuild comand issued, they will cause locking at the last stage when end latch is required.

In case they commit, there entries will go into the journal table.

Lets open another session and insert few rows and wait for 30 seconds(This is the limit i set in my script to find potential blockers)

SESSION4>insert into mytab select * from mytab where rownum<101;
100 rows created.

6. Lets check whats happening again. The journal count is still 0, since session 4 has not commited yet and the session is appearing on potential blockers list

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->enq: TX - row lock contention
JOURNAL COUNT    ---->0
STAGE            ---->Waiting for first latch.
PROGRESS         ---->No Tx started after index rebuild has commited yet. There are 0 rows in journal table
------------SESSIONS BLOCKING FIRST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2161    19807   INACTIVE                1015            0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com
------------SESSIONS THAT MAY BLOCK LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2164    36539   INACTIVE                130             0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com

7. Lets commit from session 4 and see the difference in count of journal table. It will disappear from potential blockers list as it has committed and there will be entries into the journal table

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->enq: TX - row lock contention
JOURNAL COUNT    ---->100
STAGE            ---->Waiting for first latch.
PROGRESS         ---->Few Tx started after index rebuild command have been commited and there is/are 100 rows into journal table
------------SESSIONS BLOCKING FIRST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2161    19807   INACTIVE                1134            0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com

------------SESSIONS THAT MAY BLOCK LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
NO POTENTIAL BLOCKERS FOUND FOR THE LAST LATCH

8. So far we have observed that index is stuck. Lets insert few more rows (100k approx) so that we could observe step3 else we wont able to catch the initial db file sequential read merging and check the status.

SESSION4>insert into mytab select * from mytab where rownum<100001;
100000 rows created.
SESSION4>commit;
Commit complete.

——————————-check from session 3———————-

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->enq: TX - row lock contention
JOURNAL COUNT    ---->100100
STAGE            ---->Waiting for first latch.
PROGRESS         ---->Few Tx started after index rebuild command have been commited and there is/are 100100 rows into journal table
------------SESSIONS BLOCKING FIRST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2161    19807   INACTIVE                1717            0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com
------------SESSIONS THAT MAY BLOCK LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
NO POTENTIAL BLOCKERS FOUND FOR THE LAST LATCH

9. Insert few more rows again from session 4, to ensure we block the last latch too.

SESSION4>insert into mytab select * from mytab where rownum<100;
99 rows created.

10. Lets commit from session 1 to release initial lock and check status from session 3.

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->db file sequential read
JOURNAL COUNT    ---->0
STAGE            ---->First latch acquired.db file sequential read going on. Direct path read pending after this step.
PROGRESS         ---->All Tx started before index rebuild have been commited (first latch acquired and cleared) and no entries into journal table now.

As we can see, oracle always merge the journal table entries before performing scatterd read(FTS on base table).

Once that gets completed, oracle will start scattered read. Scattered read does not need any lock, so at this time our session 4 wont lock the index rebuild. it will lock during last phase.

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->db file scattered read
JOURNAL COUNT    ---->0
STAGE            ---->First latch acquired. FTS going on
PROGRESS         ---->All Tx started before index rebuild have been commited (first latch acquired and cleared) and no entries into journal table now.

Once that is done,oracle will perform sequential read and direct path read temp to merge entries generated during scattered read and then will wait for end latch.

Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE
SQL ID IS        ---->drs3mtb5w9knb
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->enq: TX - row lock contention
JOURNAL COUNT    ---->0
STAGE            ---->Operation completed.Waiting for ending latch
PROGRESS         ---->All Tx started before index rebuild have been commited (first latch acquired and cleared) and no entries into journal table now.

------------SESSIONS BLOCKING LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2164    36539   INACTIVE                367             0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com

Once commit, index rebuild fiishes

Online Index Rebuild Stages (Parallel Clause)

Oracle performs following sequence of events with parallel clause

1. Create the journal table (inside index owner schema) SYS_JOURNAL_<objectid_index>
2. Wait for the “first latch” (enq: TX – row lock contention)
3. As soon as it gets latch, will perform , it will spawn its slave process
4. Slave process will perform db file sequential read to merge the journal table entries
5. Each slave process will perform scattered read for its bunch of blocks.
6. Slave process will again perform the sequential read to merge the journal entries which got generated during scattered read.
7. slave process job will end here here.
8. Master process will wait for the “end latch”.

Test Case

Insert few records from the first session to block index rebuild.

SESSION1>insert into mytab select * from mytab where rownum<101;
100 rows created.

Open another session and rebuild index with parallel clause

SESSION2>ALTER INDEX MYTAB_IDX REBUILD ONLINE parallel 4;

Checking the status

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE parallel 4
SQL ID IS        ---->7qr6scnthqh0f
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->enq: TX - row lock contention
JOURNAL COUNT    ---->0
STAGE            ---->Waiting for first latch.
PROGRESS         ---->No transaction started after rebuiding has committed.
------------SESSIONS BLOCKING FIRST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
2161    19807   INACTIVE                152             0               NONE            9babjv8yq8ru3           SQL*Plus        db-fc-admin-6001.iad6.amazon.com
------------SESSIONS THAT MAY BLOCK LAST LATCH ARE-------------
SID     SERIAL  STATUS          LAST_CALL_ET            COMMAND         SQL_ID          PREV_SQL_ID             MODULE          MACHINE
NO POTENTIAL BLOCKERS FOUND FOR THE LAST LATCH

As we commit the first session, parallel process will start the processing.

SESSION3>/
Enter value for sid: 2195
SQL TEXT IS      ---->ALTER INDEX MYTAB_IDX REBUILD ONLINE parallel 4
SQL ID IS        ---->7qr6scnthqh0f
JOURNAL TABLE IS ---->SYS_JOURNAL_98701
TABLE NAME IS    ---->MYTAB
EVENT_NAME IS    ---->PX Deq: Execute Reply
JOURNAL COUNT    ---->0
STAGE            ---->First latch acquired. Parallel processing going on
------------TOTAL SLAVES ARE-------------
SID     SERIAL  DEGREE          REQ_DEGREE              EVENT
2149    46710   4               4               direct path read temp
2154    22192   4               4               PX Deq: Execution Msg
2156    23451   4               4               PX Deq: Execution Msg
2157    2939    4               4               PX Deq: Execution Msg
2159    55050   4               4               direct path read temp
2168    5080    4               4               direct path read temp
2184    46569   4               4               direct path read temp
2193    3495    4               4               PX Deq: Execution Msg
------------LONGOPS SLAVES ARE-------------
SID     SERIAL  TOTALWORK               SOFAR           TIME REMAINING                  ELAPSED SECONDS         OPNAME                  EVENT
2168    5080    40531                   16913           20                              14                      Sort Output             direct path read temp
2149    46710   37403                   16918           17                              14                      Sort Output             direct path read temp
2159    55050   32739                   16872           14                              15                      Sort Output             direct path read temp
2184    46569   30720                   17012           11                              14                      Sort Output             direct path read temp