【解决】hive动态增加partitions不能超过100的问题，hivepartitions

【解决】hive动态增加partitions不能超过100的问题，hivepartitions

Author： kwu 

【解决】hive动态增加partitions不能超过100的问题，全量动态生成partitions超过100会出现如下异常：

The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to: 100

解决100限制，可设置如下参数：

set hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.max.dynamic.partitions=100000; 
SET hive.exec.max.dynamic.partitions.pernode=100000; 
insert overwrite table ods.cms_entity PARTITION (DAY)
select 
ENTITY_ID  ,
ENTITY_NAME  ,
ENTITY_DESC  ,
ENTITY_TYPE  ,
ENTITY_PID  ,
ENTITY_TIME  ,
ENTITY_PRIORITY  ,
ENTITY_STATUS  ,
ENTITY_CHANNEL  ,
ENTITY_EDITOR  ,
ENTITY_TEMPLATE  ,
ENTITY_URL    ,
ENTITY_CATEGORY  ,
ENTITY_PARAM  ,
ENTITY_SHORTNAME  ,
ENTITY_SUBTYPE  ,
ENTITY_COMPDELAY  ,
day
from stage.cms_entity_by_day;

设置动态partitions为100000，该语句插入几年按天的partitions超过1000个，成功执行语句。

版权声明：本文为博主原创文章，未经博主允许不得转载。