英文:
Oracle create table query taking too long to run
问题
我试图创建一个销售表,但是在8小时后它仍然没有运行。我尝试通过添加提示并将时间范围缩减到2022年来加快查询速度,但是在3小时后它仍在运行。有没有优化这个查询的方法?
英文:
I am trying to create a sales table however after 8 hours it still has not run. I have attempted to speed up the query by adding the hints and reducing the timeframe to 2022 only however after 3 hours it is still running. Is there a way to optimise this query?
DROP TABLE BIRTHDAY_SALES;
CREATE TABLE BIRTHDAY_SALES AS
--(
SELECT /*+ parallel(32) */
DISTINCT T.CONTACT_KEY
, S.CAMPAIGN_NAME
, S.CONTROL_GROUP_FLAG
, S.SEGMENT_NAME
, count(distinct t.ORDER_NUM) as TRANS
, count(distinct case when p.store_key = '42381' then t.ORDER_NUM else NULL end) as TRANS_ONLINE
, count(distinct case when p.store_key != '42381' then t.ORDER_NUM else NULL end) as TRANS_OFFLINE
, sum(t.ITEM_AMT) as SALES
, sum(case when p.store_key = '42381' then t.ITEM_AMT else NULL end) as SALES_ONLINE
, sum(case when p.store_key != '42381' then t.ITEM_AMT else NULL end) as SALES_OFFLINE
, sum(case when t.item_quantity_val>0 and t.item_amt<=0 then 0 else t.item_quantity_val end) QTY
, sum(case when (p.store_key = '42381' and t.ITEM_QUANTITY_VAL>0 and t.ITEM_AMT>0) then t.ITEM_QUANTITY_VAL else null end) QTY_ONLINE
, sum(case when (p.store_key != '42381' and t.ITEM_QUANTITY_VAL>0 and t.ITEM_AMT>0) then t.ITEM_QUANTITY_VAL else null end) QTY_OFFLINE
FROM CRM_TARGET.B_TRANSACTION T
JOIN BDAY_PROG S
ON T.CONTACT_KEY = S.CONTACT_KEY
JOIN CRM_TARGET.T_ORDITEM_SD P
ON T.PRODUCT_KEY = P.PRODUCT_KEY
where t.TRANSACTION_TYPE_NAME = 'Item'
and t.BU_KEY = '15'
and t.TRANSACTION_DT_KEY >= '20220101'
and t.TRANSACTION_DT_KEY <= '20221231'
and t.member_sale_flag = 'Y'
and t.bu_key = '15'
and t.CONTACT_KEY != 0
group by
T.CONTACT_KEY
, S.CAMPAIGN_NAME
, S.CONTROL_GROUP_FLAG
, S.SEGMENT_NAME
-- )
;
答案1
得分: 2
性能调优不是我们在这样的论坛上能有效处理的事情,因为有太多因素需要考虑。您将不得不检查执行计划并查看ASH数据(v$active_session_history),以查看主要的等待是什么,以及在哪个计划步骤上。只有在这之后,您才能确定出现了什么问题并采取措施来解决它。
然而,以下是一些明显需要注意的事项:
-
确保没有多对多的连接。我猜测B_TRANSACTION可能有许多具有相同CONTACT_KEY和相同PRODUCT_KEY的行。这是可以的,但然后您必须确保CONTACT_KEY在BDA_PROG内是唯一的,而PRODUCT_KEY在T_ORDITEM_SD内也是唯一的。如果不是这种情况,您将从隐藏的多对多连接中得到部分笛卡尔积,并将花费大量时间进行读写临时数据。
-
确保不超过一个连接是一对多的。多个从同一个父表派生的一对多将有效地给您提供子表之间的多对多关系,产生相同的效果。
-
您正在请求一个月的日期范围。在大多数系统中,最好执行全表扫描(如果可能的话使用并行查询),而不是使用索引获取一个月的交易数据。如果使用索引可能会导致性能问题。您可以使用提示来解决这个问题(请参见下文)。
-
在生成此类报告查询时,可能更适合使用散列连接而不是嵌套循环连接。再次强调,我只是根据您的表名猜测,只有了解您的数据才能确定这一点。
-
确保PGA工作区的大小合理。请向您的DBA查询v$pgastat并报告全局内存限制。它应该达到其最大值1G,但可能在100M以上是合理的。如果小于这个值,您可能需要请求DBA增加pga_aggregate_target,或者您可以手动设置自己的sort_area_size/hash_area_size会话参数(这不是最佳做法)。
-
您要求DOP为32。这相当高。确保数据库服务器上有足够多的CPU核心,parallel_max_servers > 64,并且没有因为某些原因被降级为串行执行。向您的DBA询问合理的DOP是多少。
-
您真的需要在ORDERNUM上使用COUNT(DISTINCT ... )吗?如果只是要计算交易数量,使用
SUM(CASE (WHEN .... ) THEN 1 ELSE 0 END)
可能会更有效率。 -
移除DISTINCT关键字。它没有起作用 - 您的GROUP BY已经确保结果是唯一的。
-
咨询ASH(v$active_session_history)以查看是否实际上被某些东西阻塞,显示某种并发等待。如果数据库配置为在添加空间之前暂停,您的CTAS可能根本不会执行任何操作,可能是由于某个库缓存锁或完整的表空间。
以下是一些尝试的建议 - 再次强调,如果不了解您的数据或表结构,这只是一个猜测:
SELECT /*+ USE_HASH(t s p) FULL(t) FULL(s) FULL(p) PARALLEL(8) */ t.contact_key . . .
英文:
Performance tuning is not something we can effectively deal with on a forum like this, as there are too many factors to consider. You will have to examine the explain plan and look at ASH data (v$active_session_history) to see what the predominant waits are and on what plan step. Only then can you determine what's wrong and take steps to fix it.
However, here are some obvious things to look for:
-
Make sure there are no many-to-many joins. I'm guessing B_TRANSACTION probably has many rows with the same CONTACT_KEY and many rows with the same PRODUCT_KEy. That's okay, but then you must ensure that CONTACT_KEY is unique within BDA_PROG and PRODUCT_KEY is unique within T_ORDITEM_SD. IF that's not the case, you will get a partial Cartesian product from the hidden many-to-many join and will spend a huge amount of time on reading/writing to temp.
-
Make sure no more than one of those joins is one-to-many. Multiple one-to-manies stemming off the same parent table will effectively give you a many-to-many between the children, with the same effect.
-
You are asking for a date range of a month. In most systems, you are better off doing a full table scan (with parallel query if you can) than using indexes to get a whole month's worth of transactional data. If it is using an index that can really mess you up. You can fix this with hints (see below)
-
It might be using nested loops joins when a reporting query like this is likely better off using hash joins. Again, I'm just guessing based on the names of your tables; only knowledge of your data can determine this for sure.
-
Ensure that the PGA workareas are of reasonable size. Ask your DBA to query v$pgastat and report the global memory bound. It should be at its max of 1G, but probably anything over 100M is reasonable. If it's less than that, you may need to ask the DBA to increase the pga_aggregate_target, or you can manually set your own sort_area_size/hash_area_size session parameters (not the best thing to do).
-
You are asking for DOP 32. That's pretty high. Ensure there are that many CPU cores on the database server, that parallel_max_servers > 64 and that you aren't getting downgraded to serial by anything. Ask your DBA what a reasonable DOP would be.
-
Do you really need COUNT(DISTINCT ... ) on ORDERNUM? If you are just counting # of transactions, it would be less work to simply say
SUM(CASE (WHEN .... ) THEN 1 ELSE 0 END)
-
Remove the DISTINCT keyword. It's not doing anything - your GROUP BY will already result in the results being distinct.
-
Consult ASH (v$active_session_history) to see if you are actually blocked by something, showing some kind of concurrency wait. Your CTAS might not be doing anything at all because of some library cache lock or full tablespace if the database is configured to suspend until space is added.
Here's something to try - again, it's a long shot without knowing your data or table structure. But I've seen enough reports like this to make at least a somewhat educated guess:
SELECT /*+ USE_HASH(t s p) FULL(t) FULL(s) FULL(p) PARALLEL(8) */ t.contact_key . . .
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论