[feature](build) Add build profiling support via DORIS_BUILD_PROFILE by shuke987 · Pull Request #61410 · apache/doris

shuke987 · 2026-03-17T03:01:44Z

Summary

Add optional build profiling to build.sh, gated by DORIS_BUILD_PROFILE environment variable
Records build metadata (user, base branch, commit, modified files, duration, exit code, load average) as JSON Lines to .build_profile.jsonl
Auto-detects base branch (master/branch-x.y/branch-selectdb-doris-x.y) by finding the closest remote main branch
Only tracks files modified since last build (mtime > last_build_time)
Zero impact when DORIS_BUILD_PROFILE is not set — no behavior change for external users

Deployment

Set in custom_env.sh on dev machines (no root required):

echo "export DORIS_BUILD_PROFILE=1" >> custom_env.sh

Test plan

Build with DORIS_BUILD_PROFILE=1, verify .build_profile.jsonl is created with correct fields
Build without DORIS_BUILD_PROFILE, verify no side effects
Interrupt build (Ctrl+C), verify record is still written with exit_code=130
Verify build failure records exit_code correctly via trap ERR

🤖 Generated with Claude Code

Add optional build profiling that records build metadata (user, base branch, commit, modified files, duration, exit code, load average) to .build_profile.jsonl. Controlled by DORIS_BUILD_PROFILE env var set in custom_env.sh, zero impact when not enabled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Thearas · 2026-03-17T03:01:49Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

shuke987 · 2026-03-17T03:03:51Z

run buildall

Print a warning and disable profiling instead of failing the build. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…le.sh Move all profiling logic from build.sh into build_profile.sh with set +e, so profiling failures can never affect the build. Use PID-based state files to avoid concurrent build conflicts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

shuke987 · 2026-03-17T03:14:40Z

run buildall

…ile.sh - Skip non _BP_* lines when reading state file to avoid eval errors - Fix load_avg regex to handle both Linux and macOS uptime output Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

shuke987 · 2026-03-17T03:23:12Z

run buildall

doris-robot · 2026-03-17T03:49:58Z

TPC-H: Total hot run time: 26787 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6cb987e7f040800b9b24d61438756384fb8a07ed, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17651	4481	4309	4309
q2	q3	10638	780	513	513
q4	4688	349	242	242
q5	7583	1188	1019	1019
q6	174	172	153	153
q7	783	829	672	672
q8	9291	1489	1357	1357
q9	4776	4704	4668	4668
q10	6314	1912	1635	1635
q11	482	250	250	250
q12	741	579	462	462
q13	18027	3005	2202	2202
q14	232	234	209	209
q15	q16	744	763	665	665
q17	757	886	406	406
q18	5814	5360	5250	5250
q19	1112	976	621	621
q20	542	476	375	375
q21	4501	1849	1428	1428
q22	346	351	425	351
Total cold run time: 95196 ms
Total hot run time: 26787 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4795	4667	4562	4562
q2	q3	3851	4425	4016	4016
q4	926	1208	790	790
q5	4032	4378	4319	4319
q6	183	177	139	139
q7	1718	1657	1528	1528
q8	2436	2753	2606	2606
q9	7673	7494	7372	7372
q10	3716	3979	3623	3623
q11	499	467	459	459
q12	507	614	491	491
q13	2989	3368	2361	2361
q14	285	303	287	287
q15	q16	772	755	716	716
q17	1145	1302	1301	1301
q18	7219	6739	6682	6682
q19	921	954	958	954
q20	2046	2121	2012	2012
q21	3904	3485	3369	3369
q22	455	421	389	389
Total cold run time: 50072 ms
Total hot run time: 47976 ms

doris-robot · 2026-03-17T04:00:43Z

TPC-DS: Total hot run time: 167847 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6cb987e7f040800b9b24d61438756384fb8a07ed, data reload: false

query5	4335	636	501	501
query6	330	225	214	214
query7	4226	469	262	262
query8	350	243	231	231
query9	8735	2733	2724	2724
query10	488	401	337	337
query11	7009	5088	4920	4920
query12	198	130	126	126
query13	1283	441	355	355
query14	5664	3686	3476	3476
query14_1	2816	2807	2815	2807
query15	196	195	177	177
query16	974	449	425	425
query17	1020	752	616	616
query18	2446	441	348	348
query19	216	207	181	181
query20	142	130	125	125
query21	214	130	113	113
query22	13245	13309	13262	13262
query23	15658	15462	15644	15462
query23_1	15903	15839	15729	15729
query24	7592	1729	1263	1263
query24_1	1292	1288	1308	1288
query25	664	556	468	468
query26	1400	285	162	162
query27	2824	506	302	302
query28	5363	2011	1936	1936
query29	846	577	466	466
query30	302	234	196	196
query31	1063	934	862	862
query32	78	68	69	68
query33	505	314	284	284
query34	890	882	510	510
query35	633	698	630	630
query36	1067	1167	1017	1017
query37	136	99	89	89
query38	2979	2955	2976	2955
query39	860	849	815	815
query39_1	782	812	797	797
query40	274	156	136	136
query41	62	59	62	59
query42	264	264	255	255
query43	251	249	225	225
query44	
query45	197	187	186	186
query46	888	998	597	597
query47	2121	2147	2075	2075
query48	300	310	224	224
query49	627	459	383	383
query50	677	275	211	211
query51	4025	4069	4028	4028
query52	257	265	255	255
query53	286	332	276	276
query54	297	271	262	262
query55	90	87	80	80
query56	313	311	321	311
query57	1933	1668	1808	1668
query58	286	279	267	267
query59	2765	2934	2751	2751
query60	343	339	327	327
query61	151	141	149	141
query62	638	589	542	542
query63	313	279	270	270
query64	5027	1286	999	999
query65	
query66	1453	436	374	374
query67	24417	24260	24240	24240
query68	
query69	420	317	281	281
query70	979	940	979	940
query71	344	307	298	298
query72	2753	2608	2383	2383
query73	539	536	311	311
query74	9626	9588	9408	9408
query75	2856	2755	2477	2477
query76	2299	1035	669	669
query77	356	376	302	302
query78	10934	11039	10478	10478
query79	3157	796	571	571
query80	1755	623	544	544
query81	575	255	223	223
query82	985	148	114	114
query83	330	275	231	231
query84	270	115	104	104
query85	904	485	446	446
query86	438	307	304	304
query87	3148	3105	3017	3017
query88	3532	2641	2632	2632
query89	417	373	345	345
query90	2034	170	167	167
query91	164	162	136	136
query92	76	72	68	68
query93	1361	846	496	496
query94	641	310	287	287
query95	573	344	373	344
query96	634	513	229	229
query97	2485	2517	2390	2390
query98	235	220	224	220
query99	1009	1010	915	915
Total cold run time: 252458 ms
Total hot run time: 167847 ms

MoanasDaddyXu

LGTM

github-actions · 2026-03-17T06:52:49Z

PR approved by anyone and no changes requested.

github-actions · 2026-03-17T06:55:08Z

PR approved by at least one committer and no changes requested.

…61410) ## Summary - Add optional build profiling to `build.sh`, gated by `DORIS_BUILD_PROFILE` environment variable - Records build metadata (user, base branch, commit, modified files, duration, exit code, load average) as JSON Lines to `.build_profile.jsonl` - Auto-detects base branch (master/branch-x.y/branch-selectdb-doris-x.y) by finding the closest remote main branch - Only tracks files modified since last build (mtime > last_build_time) - Zero impact when `DORIS_BUILD_PROFILE` is not set — no behavior change for external users ## Deployment Set in `custom_env.sh` on dev machines (no root required): ```bash echo "export DORIS_BUILD_PROFILE=1" >> custom_env.sh ```

…ILD_PROFILE #61410 (#61424) Cherry-picked from #61410 Co-authored-by: shuke <shuke@selectdb.com>

shuke987 and others added 2 commits March 17, 2026 11:07

[fix](build) Skip build profiling gracefully when python3 is unavailable

fa138fb

Print a warning and disable profiling instead of failing the build. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

[fix](build) Fix state file parsing and load_avg format in build_prof…

6cb987e

…ile.sh - Skip non _BP_* lines when reading state file to avoid eval errors - Fix load_avg regex to handle both Linux and macOS uptime output Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

shuke987 requested a review from gavinchou March 17, 2026 03:24

MoanasDaddyXu approved these changes Mar 17, 2026

View reviewed changes

shuke987 added dev/4.0.x dev/4.1.x labels Mar 17, 2026

github-actions bot added the reviewed label Mar 17, 2026

hello-stephen approved these changes Mar 17, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 17, 2026

hello-stephen merged commit 9189765 into apache:master Mar 17, 2026
34 checks passed

github-actions bot added the dev/4.0.x-conflict label Mar 17, 2026

github-actions bot mentioned this pull request Mar 17, 2026

branch-4.1: [feature](build) Add build profiling support via DORIS_BUILD_PROFILE #61410 #61424

Merged

yiguolei pushed a commit that referenced this pull request Mar 18, 2026

branch-4.1: [feature](build) Add build profiling support via DORIS_BU…

ce89c2b

…ILD_PROFILE #61410 (#61424) Cherry-picked from #61410 Co-authored-by: shuke <shuke@selectdb.com>

yiguolei added dev/4.1.0-merged and removed dev/4.1.x labels Mar 18, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature](build) Add build profiling support via DORIS_BUILD_PROFILE#61410

[feature](build) Add build profiling support via DORIS_BUILD_PROFILE#61410
hello-stephen merged 4 commits intoapache:masterfrom
shuke987:build_timer

shuke987 commented Mar 17, 2026 •

edited

Loading

Uh oh!

Thearas commented Mar 17, 2026

Uh oh!

shuke987 commented Mar 17, 2026

Uh oh!

shuke987 commented Mar 17, 2026

Uh oh!

shuke987 commented Mar 17, 2026

Uh oh!

doris-robot commented Mar 17, 2026

Uh oh!

doris-robot commented Mar 17, 2026

Uh oh!

MoanasDaddyXu left a comment

Uh oh!

github-actions bot commented Mar 17, 2026

Uh oh!

github-actions bot commented Mar 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

shuke987 commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Deployment

Test plan

Uh oh!

Thearas commented Mar 17, 2026

Uh oh!

shuke987 commented Mar 17, 2026

Uh oh!

shuke987 commented Mar 17, 2026

Uh oh!

shuke987 commented Mar 17, 2026

Uh oh!

doris-robot commented Mar 17, 2026

Uh oh!

doris-robot commented Mar 17, 2026

Uh oh!

MoanasDaddyXu left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 17, 2026

Uh oh!

github-actions bot commented Mar 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

shuke987 commented Mar 17, 2026 •

edited

Loading