Home » RDBMS Server » Performance Tuning » POWER function slows query way down? (9.2.0.8.0)
POWER function slows query way down? [message #523291] Fri, 16 September 2011 10:14 Go to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
I have a query that without a few lines that use the POWER function, will complete in 3 minutes. If I add even one line using POWER, the query slows way down and takes 4+ hours to run. Is this a problem with the basic function?, a known bug?. Is there some kind of workaround you guys can think of? Anybody got ideas on what I might do? Is there an alternative to POWER for these compound interest calculations?

Kevin

with
    rest_of_query as (
                         select
                                  a.policy_number
                                , a.caseid
                                , a.funding_method_cd
                                , a.CLMEVNTID
                                , sum(a.PAID_AMT) paid_amt
                                , a.disability_dt
--
-- problem here with POWER
-- these lines cause the query to not finish
-- goes from 3 minutes to 4+ hours
--
                                , sum(a.PAID_AMT * POWER ( 1 + Y.GAAP, a.mnths_btwn/12 )) gaap_x
                                , sum(a.PAID_AMT * POWER ( 1 + Y.PRICING, a.mnths_btwn/12 )) pricing_x
                                , sum(a.PAID_AMT * POWER ( 1 + Y.STAT, a.mnths_btwn/12 )) stat_x
                                , sum(a.PAID_AMT * POWER ( 1 + Y.UW, a.mnths_btwn/12 )) uw_x
                                , sum(a.PAID_AMT * POWER ( 1 + Y.VALUATION, a.mnths_btwn/12 )) valuation_x
                         from
                                kevtemp1 a
                              , (select * from experience.EXP_DRIVING_MONTH where rownum = 1) e
                              , experience.dlr_discount Y
                         where a.RPLCTNDT_calculation <= e.driving_date
                         and y.YEAR = TO_CHAR (a.disability_dt,'YYYY')
                         group by
                                  a.policy_number
                                , a.caseid
                                , a.funding_method_cd
                                , a.CLMEVNTID
                                , a.disability_dt
                       )
select *
from rest_of_query
/
Re: POWER function slows query way down? [message #523295 is a reply to message #523291] Fri, 16 September 2011 10:27 Go to previous messageGo to next message
Michel Cadot
Messages: 68641
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator
If you post a test case maybe we can check in our versions. Wink

By the way, as far as I can see, you use only "driving_date" column from EXP_DRIVING_MONTH, so no need of "select *". Wink
(Without speaking about the "rownum=1".)

Regards
Michel
Re: POWER function slows query way down? [message #523298 is a reply to message #523295] Fri, 16 September 2011 10:48 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
Try this for a starter. This is the simplest example I could create. Notice how a simple count goes up in cost from .04 seconds to 5 seconds. All we did was a simple calculation using POWER.

create table kevtemp9
(
    anumber number not null
  , bnumber number not null
)
/

insert into kevtemp9
select mod(rownum,100)/10000,mod(rownum,33)
from dual
connect by level <= 20000
/


select * from kevtemp9 where rownum < 10
/

select count(*) from kevtemp9
/

select count(power(1+anumber,bnumber/12)) from kevtemp9
/



Here are my timings. You can adjust the number of rows as you want.

11:44:53 SQL> create table kevtemp9
11:44:53   2  (
11:44:53   3      anumber number not null
11:44:53   4    , bnumber number not null
11:44:53   5  )
11:44:53   6  /

Table created.

Elapsed: 00:00:00.01
11:44:53 SQL> 
11:44:53 SQL> insert into kevtemp9
11:44:53   2  select mod(rownum,100)/10000,mod(rownum,33)
11:44:53   3  from dual
11:44:53   4  connect by level <= 20000
11:44:53   5  /

20000 rows created.

Elapsed: 00:00:00.04
11:44:54 SQL> 
11:44:54 SQL> 
11:44:54 SQL> select * from kevtemp9 where rownum < 10
11:44:54   2  /

   ANUMBER    BNUMBER
---------- ----------
     .0091         31
     .0092         32
     .0093          0
     .0094          1
     .0095          2
     .0096          3
     .0097          4
     .0098          5
     .0099          6

9 rows selected.

Elapsed: 00:00:00.01
11:44:54 SQL> 
11:44:54 SQL> select count(*) from kevtemp9
11:44:54   2  /

  COUNT(*)
----------
     20000

1 row selected.

Elapsed: 00:00:00.01
11:44:54 SQL> 
11:44:54 SQL> select count(power(1+anumber,bnumber/12)) from kevtemp9
11:44:54   2  /

COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
                             20000

1 row selected.

Elapsed: 00:00:05.01
11:44:59 SQL> 


Kevin
Re: POWER function slows query way down? [message #523302 is a reply to message #523298] Fri, 16 September 2011 11:19 Go to previous messageGo to next message
ThomasG
Messages: 3211
Registered: April 2005
Location: Heilbronn, Germany
Senior Member
Well, POWER of course needs more CPU than no calculation.

From 10.2.0.4:

PFK@test> select count(*) from kevtemp9
/
  COUNT(*)
----------
     20000
Elapsed: 00:00:00.04

select count(power(1+anumber,bnumber/12)) from kevtemp9
/
COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
                             20000

Elapsed: 00:00:01.38



TKPROF output:

select count(*) 
from
 kevtemp9


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.01          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        2      0.00       0.00          0         38          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        4      0.00       0.01          0         38          0           1

Misses in library cache during parse: 1
Optimizer mode: CHOOSE
Parsing user id: 42  

Rows     Row Source Operation
-------  ---------------------------------------------------
      1  SORT AGGREGATE (cr=38 pr=0 pw=0 time=2280 us)
  20000   TABLE ACCESS FULL KEVTEMP9 (cr=38 pr=0 pw=0 time=114 us)

********************************************************************************

select count(power(1+anumber,bnumber/12)) 
from
 kevtemp9


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        2      1.37       1.34          0         38          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        4      1.37       1.34          0         38          0           1

Misses in library cache during parse: 1
Optimizer mode: CHOOSE
Parsing user id: 42  

Rows     Row Source Operation
-------  ---------------------------------------------------
      1  SORT AGGREGATE (cr=38 pr=0 pw=0 time=1347071 us)
  20000   TABLE ACCESS FULL KEVTEMP9 (cr=38 pr=0 pw=0 time=36 us)


The funny thing is this:

declare 
 i number;
 x number;
begin

for i in 1..20000 loop 
x := power (i,10);
end loop; 
end;
/ 

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.05


declare 
 i number;
 x number;
begin

for i in 1..20000 loop 
x := power (i,10.54146415412);
end loop; 
end;
/ 

PL/SQL procedure successfully completed.

Elapsed: 00:00:03.09


So it seems power needs longer the more complicated the numbers get.
Re: POWER function slows query way down? [message #523304 is a reply to message #523302] Fri, 16 September 2011 11:42 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
Yes, POWER does require more CPU than not doing it. But that much more? WOW!

Thanks for the TKPROF. It shows well the issue.

And I too saw that POWER is even more expensive than normal when given a fractional exponent.

But this leaves me with my original question... Anyone got ideas on how to get what I need without the performance penalty?

Thanks for your work Thomas.

Kevin
Re: POWER function slows query way down? [message #523306 is a reply to message #523304] Fri, 16 September 2011 11:49 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
>But this leaves me with my original question... Anyone got ideas on how to get what I need without the performance penalty?
JAVA?
Re: POWER function slows query way down? [message #523311 is a reply to message #523306] Fri, 16 September 2011 12:13 Go to previous messageGo to next message
ThomasG
Messages: 3211
Registered: April 2005
Location: Heilbronn, Germany
Senior Member
He. Would be nice to know if the "build in" power function is completely C++, or if it's already implemented in PL/SQL.

Is it perhaps possible to pull the power call further up in the sum? Basically

sum(a * power(x,y))

be the same as

sum(a) * power (x,y)

IF x and y stay the same. Don't know if that is the case in the data model, though.

Re: POWER function slows query way down? [message #523312 is a reply to message #523311] Fri, 16 September 2011 12:18 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
It is, and in many cases that would help somewhat. But...

1) it turns out that the grouping in my original query is really meaningless. It does nothing. Same number of rows in and out. I am just following the original code. So moving it around in this case does not reduce the number of rows on which the calculations are done.

2) it still does not address the core issue that POWER is very expensive it seems.

But it was very astute of you to suggest this. Most people I work with would not be of the level to see that so this tells me your skill level is pretty high.

I was hoping someone with a math background could offer up an alternative formula.

Or that someone in the Oracle space would know of a patch, or an Oracle supplied utility or DBMS package that would offer an alternative.

Kevin
Re: POWER function slows query way down? [message #523332 is a reply to message #523312] Fri, 16 September 2011 14:44 Go to previous messageGo to next message
ThomasG
Messages: 3211
Registered: April 2005
Location: Heilbronn, Germany
Senior Member
Thanks for the flowers.

I just tried it doing a power calculation in perl, to have some sort of comparison.

tmp > cat test.pl 
#!/bin/perl

for ( $i =0; $i < 20000 ; $i++ ) {
  $e = $i ** 10.33535334646;
}
tmp > time perl test.pl

real	0m0.008s
user	0m0.003s
sys	0m0.003s


So it really might indeed be worthwhile to try if a java stored procedure that does a power is faster than the Oracle build in one. Since it really seems to be quite slow compared to other implementations on the same machine.

Or write an intermediate file and do the power calculation in perl.
Re: POWER function slows query way down? [message #523340 is a reply to message #523291] Fri, 16 September 2011 15:41 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
Thanks everyone. I think I am all set. Here is an even smaller test case. I think my solution is simply to wait for the upgrade to 11gR2. I am headed for EXADATA in a couple of weeks and tests there are very encouraging. I even ran the original query on EXADATA and got the expected timings, not the forever timings.

9i database
SQL> with
  2       some_data as (
  3                      select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
  4                      from dual
  5                      connect by level <= 20000
  6                    )
  7  select count(*) from some_data
  8  /

  COUNT(*)
----------
     20000

1 row selected.

Elapsed: 00:00:00.02
SQL> with
  2       some_data as (
  3                      select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
  4                      from dual
  5                      connect by level <= 20000
  6                    )
  7  select count(power(1+anumber,bnumber/12)) from some_data
  8  /

COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
                             20000

1 row selected.

Elapsed: 00:00:05.03
11g EXADATA2
SQL> with
  2       some_data as (
  3                      select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
  4                      from dual
  5                      connect by level <= 20000
  6                    )
  7  select count(*) from some_data
  8  /

  COUNT(*)
----------
     20000

1 row selected.

Elapsed: 00:00:00.01
SQL> with
  2       some_data as (
  3                      select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
  4                      from dual
  5                      connect by level <= 20000
  6                    )
  7  select count(power(1+anumber,bnumber/12)) from some_data
  8  /

COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
                             20000

1 row selected.

Elapsed: 00:00:00.04


Quote:
ALL HAIL THE EXADATA


As usual, OraFAQ has been faster than Oracle Service Requests.

Kevin
Re: POWER function slows query way down? [message #523364 is a reply to message #523340] Sat, 17 September 2011 02:54 Go to previous messageGo to next message
John Watson
Messages: 8929
Registered: January 2010
Location: Global Village
Senior Member
Kevin, is it possible for you to run the query again on the Exadata DB Machine? It would be interesting to see if the performance is because the cell offload processing is working, or if it is merely because Exadata can serve blocks more quickly than conventional storage. I have found several circumstances where offload processing is not as efficient as it should be.

Are you familiar with how to get this information?

John.
Re: POWER function slows query way down? [message #523369 is a reply to message #523364] Sat, 17 September 2011 04:17 Go to previous messageGo to next message
John Watson
Messages: 8929
Registered: January 2010
Location: Global Village
Senior Member
If you have the time, Kevin, what I would be interested in seeing is the results of a test something like the following, which tries to confirm whether the POWER function is in fact being evaluated more efficiently by the Exadata layer than it can be evaluated by the DB layer. Your performance improvement could be due to so many things: the 11.2 upgrade, Exadata offload processing, better DB server hardware, the Exadata storage, anything. I don't know that this test is valid, but it might be interesting. I'm trying to isolate some of the factors.

--make sure we do direct reads
alter session set "_serial_direct_read"=always

--check that the power function is offloadable
select offloadable from v$sqlfn_metadata where name='POWER';

--display whether the step COULD (not necessarily WOULD !) be offloaded
alter session set cell_offload_plan_display=always;
set autotrace on exp
select count(power(1+anumber,bnumber/12)) from kevtemp9;
select count(*) from kevtemp9;
set autotrace off

--check the timings with offload
set timing on
alter system flush shared_pool;
alter system flush buffer_cache;
alter session set cell_offload_processing=true;
select count(*) from kevtemp9;
alter system flush buffer_cache;
select count(power(1+anumber,bnumber/12)) from kevtemp9;

--did we actually get any offload?
select 
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(power(1+anumber,bnumber/12)) from kevtemp9';

select 
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(*) from kevtemp9';


--and now without offload
alter session set cell_offload_processing=false;
alter system flush shared_pool;
alter system flush buffer_cache;
select count(*) from kevtemp9;
alter system flush buffer_cache;
select count(power(1+anumber,bnumber/12)) from kevtemp9;


--did we actually get any offload?
select 
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(power(1+anumber,bnumber/12)) from kevtemp9';

select 
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(*) from kevtemp9';
Re: POWER function slows query way down? [message #523400 is a reply to message #523291] Sat, 17 September 2011 13:27 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
I'll run the tests soon as I get a chance.

But my expectation is that this has nothing to do with blocks of data. That is why I created the second simpler test example. Using the second sql query that does not reference any actual table, I figure 20000 rows constructed on the fly of a wicked short length would mean no disk was ever accessed and thus EXADATA's block serving ability would mean nothing.

I agree, there could be many reasons for the difference and it would be nice to have some idea of where the benefit comes from.

Kevin
Re: POWER function slows query way down? [message #523407 is a reply to message #523400] Sat, 17 September 2011 14:42 Go to previous messageGo to next message
John Watson
Messages: 8929
Registered: January 2010
Location: Global Village
Senior Member
I guess I didn't explain properly what I'm thinking.
The evaluation of the POWER function should be offloadable to the Exadata tier, so the database won't actually evaluate it at all: the data returned by the Exadata cell will be the final value, with the power already computed. The Exadata software is not an Oracle instance, so it would seem likely that the implementation of the power function is totally different. That MIGHT be why the performance is so much better.
But what I'm also thinking is that you could get a very different result with a more complex example where the computation of the power can't always be offloaded. There are a worrying number of circumstances (which can change from one run of the query to another) where offload can't be done, which can give seriously erratic performance. So I was hoping to determine just where the power is being evaluated.
If you can run my test, or variations thereof, I would really like to see the results.

(ps - I've just ordered a copy of this,
http://www.amazon.co.uk/Expert-Oracle-Exadata-Kerry-Osborne/dp/1430233923
I have a colleague who knows the guys that wrote it, and he tells me it is excellent)
Re: POWER function slows query way down? [message #523433 is a reply to message #523407] Sun, 18 September 2011 11:51 Go to previous message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
OK, I just bought it. 76$. Hoping it is more than just a recapitulation of the manuals. thanks buddy.

And I see what you are getting at. You are not telling me that the thing went faster on EXADATA. You are pointing at that if EXADATA did to the work, then I could someday end up with performance tanking on me in other situations where the work is not passed off to EXADATA. That is a good thing to point out.

Let me get on running the stuff you provided.

Kevin

OOPs just checked EBAY. There is a US seller offering it delievered for 25$less. Oh well...

[Updated on: Sun, 18 September 2011 11:55]

Report message to a moderator

Previous Topic: query taking long time
Next Topic: Need help to understand in depth about ORA-01555
Goto Forum:
  


Current Time: Fri Apr 19 20:54:15 CDT 2024