Skip to content

638 倍嗰個數唔係 page latency

638 倍嗰個數唔係 page latency

本文由 AI 從英文翻譯。 閱讀英文原文

兩個數,兩個唔同 workload,兩個都唔係量 page latency。嗰個 638 倍,係喺一個啱啱 create 嘅 file 上面做一次 stat(即係 file metadata lookup),EFS 對比 block device。嗰個 4 倍,係一個 synthetic、request 形狀嘅 sweep 打去 local volume,mean aggregate stat time;但 median 嘅差距其實得 1.2 到 1.3 倍左右。Mean 係真嘅,不過佢始終話唔到你邊個 request 慢。

文入面每個 finding 都會用返佢個名去引;全部都喺 study repository 度,連背後啲 run 都喺度。

把尺:需求係固定嘅

擺 WordPress 喺度,係因為佢代表咗大部分 estate 都係由呢一類嘢砌成:喺 object storage 變成常態之前寫落嘅 software,佢假設有個 POSIX filesystem,而佢自己嘅 core code、plugin、theme、template hierarchy 就不斷去問個 filesystem。呢個 study 嘅前提係:當個 application 係黑盒。我會改 task definition、mount topology、container image、php.ini 同 opcache、仲有 proxy;但我不會改個 application。呢個 architecture 嘅 naive 版本我六年前砌過,喺度佢只係 provenance,唔係研究對象。

下面所有數字都係 per warm request。一個 warm request 大約出 4,300 個 filesystem syscall,當中大約 3,900 個係 stat;我試過嘅 php.ini profile 冇一個推得低呢個底(warm-request syscall census,E0-n10)。Cold request 係另一個數,15,400 到 16,400 個 operation,唔會加落 warm 個數度。

量度項目數目
Home / product / wp-admin,總 syscall5,157 / 5,265 / 5,140
有真實 item 嘅 cart 同 checkout,總 syscall5,941 / 5,884
wp-includes(core),filesystem op大約 103
wpforms-lite,filesystem op大約 1,155
Elementor / WooCommerce / theme / ACF590 / 528 / 214 / 137
ENOENT lookup大約 1,600

endpoint 嗰兩行係總 syscall,唔係 filesystem call,所以先會高過 census 個數;佢哋喺 content 同 WooCommerce page 之間差 14 到 15%,但重複十次都完全唔變(cart 同 checkout 個數,E0-cart-checkout)。filesystem call 入面大約 1,600 個係回 ENOENT,即係個 application 喺度問「有冇呢個 file」。喺 block device 度呢條問題幾乎唔使錢;喺 network filesystem 度,就係用 latency 去換一個「乜都學唔到」嘅答案。呢場風暴係嚟自 plugin code 而唔係 core,係逐個 component 觀察到嘅。至於每個 probe 點解存在,係推斷返嚟,因為冇一個 call site 追溯到 PHP source。

大家喺呢度成日 recommend 嘅兩個設定,改嘅係 application 幾密去問,唔係問幾多。熄咗 opcache.validate_timestamps,喺 revalidation window 入面大約慳返 700 個 open call,同大約 1.8% 嘅 stat;而開住 validating 嘅 profile,每個 revalidate_freq window 就要畀一次大約 +4,400 個 stat 嘅稅,naive 係 2 秒,tune 完係 60 秒(同一個 census,E0-n10)。我跑過嘅三個 profile,包括一個 realpath_cache 32 MB、TTL 3600 秒嘅,個底全部都係留喺大約 3,900(PHP profile 對比,E0-first-run)。因為呢筆稅係每個 window 畀一次,所以喺低 traffic 嘅 site,佢 per request 嘅價值最高。

兩個數,兩個 workload

每次呼叫 · 一個 stat,新檔案

1101001000每次呼叫 · 一個 stat,新檔案 — replication 1 of 1: 638.2×638.2×

638.2×

每個 request · 中位數

1101001000每個 request · 中位數 — replication 1 of 3: 1.21×1.21×每個 request · 中位數 — replication 2 of 3: 1.29×1.29×每個 request · 中位數 — replication 3 of 3: 1.25×1.25×

1.21× / 1.29× / 1.25×

每個 request · 平均值

1101001000每個 request · 平均值 — replication 1 of 3: 3.91×3.91×每個 request · 平均值 — replication 2 of 3: 4.43×4.43×每個 request · 平均值 — replication 3 of 3: 5.19×5.19×

3.91× / 4.43× / 5.19×

每個 request · p99

1101001000每個 request · p99 — replication 1 of 3: 88.7×88.7×每個 request · p99 — replication 2 of 3: 125.9×125.9×每個 request · p99 — replication 3 of 3: 87.6×87.6×

88.7× / 125.9× / 87.6×

每次呼叫 · 一個 stat,新檔案

1101001000每次呼叫 · 一個 stat,新檔案 — replication 1 of 1: 638.2×638.2×

638.2×

每個 request · 中位數

1101001000每個 request · 中位數 — replication 1 of 3: 1.21×1.21×每個 request · 中位數 — replication 2 of 3: 1.29×1.29×每個 request · 中位數 — replication 3 of 3: 1.25×1.25×

1.21× / 1.29× / 1.25×

每個 request · 平均值

1101001000每個 request · 平均值 — replication 1 of 3: 3.91×3.91×每個 request · 平均值 — replication 2 of 3: 4.43×4.43×每個 request · 平均值 — replication 3 of 3: 5.19×5.19×

3.91× / 4.43× / 5.19×

每個 request · p99

1101001000每個 request · p99 — replication 1 of 3: 88.7×88.7×每個 request · p99 — replication 2 of 3: 125.9×125.9×每個 request · p99 — replication 3 of 3: 87.6×87.6×

88.7× / 125.9× / 87.6×

Log₁₀ scale · ratio · Supplied replications

同一個比較,兩個分母。每次呼叫:638.2×,量度喺剛建立嘅檔案上面。每個 request:同一個檔案系統喺貼近真實嘅存取模式下,中位數 1.21× / 1.29× / 1.25×、平均值 3.91× / 4.43× / 5.19×、p99 88.7× / 125.9× / 87.6×(每個 replication 一個值)。對數刻度;run results/E2/20260910T163206Z-66dcff8, results/E5-formal/20261028T160106Z-dd5fce0。

Source: mikeng-io/aws-wordpress@b52192b
results/E2/20260910T163206Z-66dcff8 · results/E5-formal/20261028T160106Z-dd5fce0

Measurement definitions / Command
每次呼叫 · 一個 stat,新檔案
EC2 EFS direct TLS stat p50 / EC2 instance-store stat p50, pooled across three replications. Formula: raw numerator / raw denominator in ns, before display rounding; no ratio field exists in the pinned summary. Source JSON pointers (numerator, denominator): /tiers/ec2~1efs/stat/p50_ns; /tiers/ec2~1instance_store/stat/p50_ns.
每個 request · 中位數
EFS plain per-request summed stat time median / matched-EBS median, delta=1 s. Leaves are numerator, denominator pairs per replication in chronological order of the three eligible formal-v3 replications. Formula: raw numerator / raw denominator in ns, before display rounding; no ratio field exists in the pinned summary. Source JSON pointers (numerator, denominator): /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_per_request_ns/median; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/ebs_per_request_ns/median; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_per_request_ns/median; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/ebs_per_request_ns/median; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_per_request_ns/median; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/ebs_per_request_ns/median.
每個 request · 平均值
Reported amortised per-request stat-time EFS plain / EBS ratios, delta=1 s; the array is the three eligible formal-v3 replications in chronological order. Source JSON pointers (output order): /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_over_ebs; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_over_ebs; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_over_ebs.
每個 request · p99
EFS plain per-request summed stat time p99 / matched-EBS p99, delta=1 s. Leaves are numerator, denominator pairs per replication in chronological order of the three eligible formal-v3 replications. Formula: raw numerator / raw denominator in ns, before display rounding; no ratio field exists in the pinned summary. Source JSON pointers (numerator, denominator): /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_per_request_ns/p99; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/ebs_per_request_ns/p99; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_per_request_ns/p99; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/ebs_per_request_ns/p99; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_per_request_ns/p99; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/ebs_per_request_ns/p99.
python3 analysis/export_figures.py

喺 EFS 上面做一次 stat,時間係喺 block device 度做同一個 call 嘅 638 倍。但係喺一個 request 形狀嘅 sweep 入面,啲 file 至少兩分鐘前已經存在,一個 client 每秒逐個 stat 一次,aggregate stat time 對比打去 local EBS 嘅同一個 sweep:median 大約 1.2 到 1.3 倍,mean 大約 3.9 到 5.2 倍。將呢兩個 ratio 乘埋一齊,就會得出一個冇人量過嘅數——我自己之前一稿 findings 就係咁做;而且 per-request 個結果都冇推翻 per-call 嗰個,只係將佢嘅適用範圍收窄。你見到有人單獨引用嗰個 638 倍,就係我舊嗰稿。兩個 sweep 用嘅係 synthetic tree,唔係 WordPress 嘅 request path,而且呢度冇一樣嘢係 scale 到上面講嘅 3,900 個 call census。

Tierstat p50open+read p50create p50unlink p50
Instance store1.3 µs5.2 µs19.7 µs7.5 µs
EBS gp31.3 µs5.3 µs19.9 µs7.6 µs
Fargate ephemeral1.8 µs5.7 µs21.9 µs11.4 µs
EC2 + EFS,Elastic throughput / One Zone / General Purpose / NFS 4.1814 µs904 µs7.34 ms3.00 ms
Fargate + EFS,同一個 configuration778 µs679 µs6.99 ms2.93 ms

pooled 自三個 replication,每組三個獨立 deployment(per-call tier matrix,E2-block-backed)。全文嘅 ratio 都係用未四捨五入嘅 pooled p50 計,所以用呢個四捨五入咗嘅表再計返,會得出 626 倍而唔係 638 倍。兩個 block substrate 喺度分唔開:instance store 對 EBS,stat 1.00 倍、open+read 1.03 倍、create 1.01 倍、unlink 1.02 倍,p99 都係咁樣行。一過咗條界就完全唔埋身:stat 638 倍、unlink 399 倍、create 373 倍、open+read 174 倍。metadata 類 operation 受嘅苦,遠比細 file 嗰個 open+read pass 多,而呢個就係問題嘅形狀:server-backed 嘅 tier 做呢種 read,比佢答「個 file 點樣」嘅問題好得多。呢個唔係 throughput test,我唔會喺度讀出任何關於搬 byte 嘅結論。

係 protocol 而唔係 hardware——呢個係我嘅推斷,唔係一個獨立跑出嚟嘅結果:冇一個 run 係只變 protocol,所以佢係靠三個 block substrate 講同一件事、再加六個 EFS configuration 講同一件事撐住。四個候選 lever 量出嚟都係 noise,而 replication 之間嘅底已經係 1.49 到 1.82 倍:經 local efs-proxy 行 transit encryption(視乎 mount path,1.00 到 1.10 倍)、mount topology(1.00 到 1.09 倍,而 ECS-managed 嗰條 path 其實叫嘅係同一個 mount helper,同手寫嗰個一樣)、EFS path 上 EC2 對 Fargate(stat 1.06 倍、create 1.01 倍)、仲有六個 EFS configuration,stat p50 由 683 到 786 µs,1.15 倍嘅差距,而逐個 configuration 對 block group 嘅差距係 357 到 630 倍。Pooled 嘅 638 倍係 headline ratio;357 到 630 倍係每個獨立 configuration 自己做到嘅數。呢啲全部係同一把尺上面嘅 per-call finding,冇一個係證實咗嘅 tail lever。

有兩個 collection 要分開講:上面 Fargate ephemeral 嗰行係嚟自 tier matrix,而另一個獨立 collection 量咗三個 Fargate task、4,020 個 sample(Fargate ephemeral finding,E3-fargate-ephemeral)。再獨立一樣嘢:EFS 係每個 task mount 一次,唔係每個 host 一次。同一個 t4g.small 上面兩個 task,同一個 task definition、同一個 filesystem,出咗兩個 NFS4 client mount、兩個 efs-proxy process,同埋各自嘅 attribute cache(mount isolation finding,E1-mount-per-task)。同機 co-locate 嘅 task 默認係唔會幫對方 warm cache 嘅,所以一個 task 可以肯定自己本地揸住嘅 code,就只有佢自己寫落本地嗰啲。刻意 share host mount 嗰個情況我冇量過,所以兩邊我都不作判斷。

File 幾舊、cache 歷史、同個 timer

EFS

07.173EFS — replication 1 of 3: 7.173 s7.173 sEFS — replication 2 of 3: 6.502 s6.502 sEFS — replication 3 of 3: 7.122 s7.122 s

7.173 s / 6.502 s / 7.122 s

FSx OpenZFS

07.173FSx OpenZFS — replication 1 of 3: 3.385 s3.385 sFSx OpenZFS — replication 2 of 3: 3.408 s3.408 sFSx OpenZFS — replication 3 of 3: 3.416 s3.416 s

3.385 s / 3.408 s / 3.416 s

FSx ONTAP

07.173FSx ONTAP — replication 1 of 3: 0.9938 s0.9938 sFSx ONTAP — replication 2 of 3: 0.9682 s0.9682 sFSx ONTAP — replication 3 of 3: 0.9875 s0.9875 s

0.9938 s / 0.9682 s / 0.9875 s

EFS

07.173EFS — replication 1 of 3: 7.173 s7.173 sEFS — replication 2 of 3: 6.502 s6.502 sEFS — replication 3 of 3: 7.122 s7.122 s

7.173 s / 6.502 s / 7.122 s

FSx OpenZFS

07.173FSx OpenZFS — replication 1 of 3: 3.385 s3.385 sFSx OpenZFS — replication 2 of 3: 3.408 s3.408 sFSx OpenZFS — replication 3 of 3: 3.416 s3.416 s

3.385 s / 3.408 s / 3.416 s

FSx ONTAP

07.173FSx ONTAP — replication 1 of 3: 0.9938 s0.9938 sFSx ONTAP — replication 2 of 3: 0.9682 s0.9682 sFSx ONTAP — replication 3 of 3: 0.9875 s0.9875 s

0.9938 s / 0.9682 s / 0.9875 s

Linear scale · s · Supplied replications

喺每個 tier 建立測試檔案所需嘅秒數,每個 replication 一點:EFS 係 7.173 s / 6.502 s / 7.122 s,對比 ONTAP 嘅 0.9938 s / 0.9682 s / 0.9875 s。下一張圖會顯示呢件事對 benchmark 造成咩影響。Run results/E2/20261015T000959Z-85382cc。

Source: mikeng-io/aws-wordpress@b52192b
results/E2/20261015T000959Z-85382cc

Measurement definitions / Command
EFS
duration of the 1,000-file create pass, per replication; derived by analysis/e2_cache_window.py from the pinned FSx run's raw CSVs; the array is the three replications. Source JSON pointers (output order): /tiers/efs/replications/rep-1/create_pass_s; /tiers/efs/replications/rep-2/create_pass_s; /tiers/efs/replications/rep-3/create_pass_s.
FSx OpenZFS
duration of the 1,000-file create pass, per replication; derived by analysis/e2_cache_window.py from the pinned FSx run's raw CSVs; the array is the three replications. Source JSON pointers (output order): /tiers/fsx_openzfs/replications/rep-1/create_pass_s; /tiers/fsx_openzfs/replications/rep-2/create_pass_s; /tiers/fsx_openzfs/replications/rep-3/create_pass_s.
FSx ONTAP
duration of the 1,000-file create pass, per replication; derived by analysis/e2_cache_window.py from the pinned FSx run's raw CSVs; the array is the three replications. Source JSON pointers (output order): /tiers/fsx_ontap/replications/rep-1/create_pass_s; /tiers/fsx_ontap/replications/rep-2/create_pass_s; /tiers/fsx_ontap/replications/rep-3/create_pass_s.
python3 analysis/export_figures.py

per-call benchmark 會 create 一批 file,然後幾秒之後去 stat 佢哋;呢把尺行一 pass,就係一個 create sweep 跟住一個 stat sweep。FSx 嗰組就揭示咗呢樣嘢其實量緊乜:喺每一個 NFS tier 上面,快嘅 stat 都係集中喺最近 create 嗰批 file,形成一條連續嘅尾,去到一個固定嘅 age 就斷(server-backed tier matrix,E2-server-backed)。

Tier,pinned configurationcreate 呢批 file 用幾耐有幾多 stat 係快答邊界 age
EFS,Elastic / One Zone / General Purpose / NFS 4.16.5 到 7.2 秒31 到 38%2.30 到 2.48 秒
FSx for OpenZFS,Single-AZ 64 GiB,128 MBps3.4 秒68 到 70%2.91 到 2.92 秒
FSx for ONTAP,Single-AZ gen-1 1024 GiB,128 MBps1.0 秒100%成個 sweep 都喺 window 入面

create sweep 越快,stat sweep 開始嗰陣就越多 file 仲喺有效 window 入面,所以越多 stat 可以本地答。ONTAP 1,000 個 stat call 只係出咗 8 個 GETATTR RPC,而佢喺同一組度嘅 1.7 µs p50 係 cache hit,唔係一個 round trip。呢個數係屬於佢 Single-AZ gen-1、1024 GiB、128 MBps 嗰個 provisioning point,唔係 ONTAP 成個 service 嘅特性。

條界係 Linux client 嘅 acregmin,默認 3 秒(nfs(5)),而個 backoff 係喺 kernel v6.1 嘅 fs/nfs/inode.c 度讀出嚟,唔係從呢啲 run 推返嚟。每次 revalidation 見到冇變,timeout 就 double:3 → 6 → 12 → 24 → 48 → 60 秒,封頂喺 acregmax,一有變就 reset。所以,唔係「舊過 60 秒嘅 file」就會 miss:個 timer 係跟住呢個 client 上面嗰個 inode 嘅 revalidation 歷史行,所以一個一年前寫落嘅 PHP file,喺 cold mount 上面第一次 lookup,都係同一個全新 file 一樣由 3 秒開始。

ServerLinux NFS clientApplicationServerLinux NFS clientApplicationfirst touch on a cold clientattributes valid, up to 60 svalidity elapsesopt[read delegation held, OpenZFScapability only]statLOOKUP compoundattributes, timer starts at acregminansweredstat againanswered from memory, no RPCstat after expiryGETATTRunchanged, timeout doubles toward acregmaxansweredstatanswered locally, timer never runs

咁就得出兩種唔同嘅 miss workload。喺 cold client 上面嘅 first touch——即係 task 開頭、scale-out、或者 deploy 之後——要畀一個 LOOKUP compound,量出嚟係 EFS 嗰幾條 arm 上面 revalidation 成本嘅 2.8 到 4.4 倍;OpenZFS 上面係 1.04 到 1.08 倍,其中一個 replication 係 0.72 倍;ONTAP 上面係 1.08 到 1.44 倍(first-touch 同 revalidation 對比,E5-attribute-cache)。嗰個大 premium 係 EFS 觀察到嘅嘢,唔係 NFS 嘅本性。Steady-state revalidation 每個 file 大約每 acregmax 畀一次 GETATTR:EFS 263 到 357 µs,OpenZFS 69 到 98 µs,ONTAP 109 到 111 µs。呢度要修正我自己之前一個數:per-call benchmark 嗰個 827 µs 嘅 EFS miss 成本,喺呢度重現唔到;呢啲數係佢嘅 0.32 到 0.43 倍,因為嗰個係另一種 miss。

個 timer 到底停唔停得,係 protocol 嘅能力問題。EFS 唔可以發 NFSv4 delegation,所以 OPEN 永遠回 OPEN_DELEGATE_NONE(EFS limits);FSx for OpenZFS 可以發 read delegation(OpenZFS performance guide),啲 run 喺佢身上數到 1,000 個 DELEGRETURN,而 EFS 同 ONTAP 係零。Client 揸住一個 delegation,就可以本地答 stat,連 timer 都唔使起。呢個只係「揸到 delegation 嘅能力」,唔係話 OpenZFS 每次 access 都揸住一個。

呢個機制係有 doc 同 kernel 撐住嘅推斷,而且啱啱好對得上量到嘅邊界;要真正 settle,就要改 actimeo 再睇條界會唔會跟住郁。用 20 µs 做 cutoff、當一個 stat 係「client memory 答」嗰條,係事後(post-hoc)加嘅描述性規則。而個缺陷係出喺個 metric 嘅範圍,唔係出喺 benchmark 作者身上:喺啱啱 create 嘅 file 上面做 per-call 量度,等於喺 cache lifecycle 唔同嘅位去比較唔同 tier——EFS 得 31 到 38% 嘅 stat、OpenZFS 得 68 到 70% 落喺 window 入面,ONTAP 就全部落喺入面。佢量到嘅係一個受 create 歷史影響嘅 hit/miss 混合物,唔係一個均勻 warm 嘅 cache。

心跳:到期之前幾乎打和

  • EFS · 每個 request 嘅 stat 時間 ×3
  • EBS 對照組 · 每個 request 嘅 stat 時間 ×3
1101001000 EFS · 每個 request 嘅 stat 時間 — median: 2.14 ms; mean: 6.942 ms; p99: 164.1 ms; max: 233.3 ms EFS · 每個 request 嘅 stat 時間 — median: 2.14 ms 2.14 ms EFS · 每個 request 嘅 stat 時間 — mean: 6.942 ms 6.942 ms EFS · 每個 request 嘅 stat 時間 — p99: 164.1 ms 164.1 ms EFS · 每個 request 嘅 stat 時間 — max: 233.3 ms 233.3 ms EFS · 每個 request 嘅 stat 時間 — median: 2.091 ms; mean: 7.221 ms; p99: 216.6 ms; max: 244.1 ms EFS · 每個 request 嘅 stat 時間 — median: 2.091 ms 2.091 ms EFS · 每個 request 嘅 stat 時間 — mean: 7.221 ms 7.221 ms EFS · 每個 request 嘅 stat 時間 — p99: 216.6 ms 216.6 ms EFS · 每個 request 嘅 stat 時間 — max: 244.1 ms 244.1 ms EFS · 每個 request 嘅 stat 時間 — median: 1.875 ms; mean: 7.947 ms; p99: 151.3 ms; max: 262.4 ms EFS · 每個 request 嘅 stat 時間 — median: 1.875 ms 1.875 ms EFS · 每個 request 嘅 stat 時間 — mean: 7.947 ms 7.947 ms EFS · 每個 request 嘅 stat 時間 — p99: 151.3 ms 151.3 ms EFS · 每個 request 嘅 stat 時間 — max: 262.4 ms 262.4 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.773 ms; mean: 1.775 ms; p99: 1.849 ms; max: 3.454 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.773 ms 1.773 ms EBS 對照組 · 每個 request 嘅 stat 時間 — mean: 1.775 ms 1.775 ms EBS 對照組 · 每個 request 嘅 stat 時間 — p99: 1.849 ms 1.849 ms EBS 對照組 · 每個 request 嘅 stat 時間 — max: 3.454 ms 3.454 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.627 ms; mean: 1.63 ms; p99: 1.72 ms; max: 2.932 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.627 ms 1.627 ms EBS 對照組 · 每個 request 嘅 stat 時間 — mean: 1.63 ms 1.63 ms EBS 對照組 · 每個 request 嘅 stat 時間 — p99: 1.72 ms 1.72 ms EBS 對照組 · 每個 request 嘅 stat 時間 — max: 2.932 ms 2.932 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.501 ms; mean: 1.533 ms; p99: 1.727 ms; max: 2.763 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.501 ms 1.501 ms EBS 對照組 · 每個 request 嘅 stat 時間 — mean: 1.533 ms 1.533 ms EBS 對照組 · 每個 request 嘅 stat 時間 — p99: 1.727 ms 1.727 ms EBS 對照組 · 每個 request 嘅 stat 時間 — max: 2.763 ms 2.763 ms medianmeanp99max

EFS · 每個 request 嘅 stat 時間 — median: 2.14 ms / mean: 6.942 ms / p99: 164.1 ms / max: 233.3 ms

EFS · 每個 request 嘅 stat 時間 — median: 2.091 ms / mean: 7.221 ms / p99: 216.6 ms / max: 244.1 ms

EFS · 每個 request 嘅 stat 時間 — median: 1.875 ms / mean: 7.947 ms / p99: 151.3 ms / max: 262.4 ms

EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.773 ms / mean: 1.775 ms / p99: 1.849 ms / max: 3.454 ms

EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.627 ms / mean: 1.63 ms / p99: 1.72 ms / max: 2.932 ms

EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.501 ms / mean: 1.533 ms / p99: 1.727 ms / max: 2.763 ms

1101001000 EFS · 每個 request 嘅 stat 時間 — median: 2.14 ms; mean: 6.942 ms; p99: 164.1 ms; max: 233.3 ms EFS · 每個 request 嘅 stat 時間 — median: 2.14 ms 2.14 ms EFS · 每個 request 嘅 stat 時間 — mean: 6.942 ms 6.942 ms EFS · 每個 request 嘅 stat 時間 — p99: 164.1 ms 164.1 ms EFS · 每個 request 嘅 stat 時間 — max: 233.3 ms 233.3 ms EFS · 每個 request 嘅 stat 時間 — median: 2.091 ms; mean: 7.221 ms; p99: 216.6 ms; max: 244.1 ms EFS · 每個 request 嘅 stat 時間 — median: 2.091 ms 2.091 ms EFS · 每個 request 嘅 stat 時間 — mean: 7.221 ms 7.221 ms EFS · 每個 request 嘅 stat 時間 — p99: 216.6 ms 216.6 ms EFS · 每個 request 嘅 stat 時間 — max: 244.1 ms 244.1 ms EFS · 每個 request 嘅 stat 時間 — median: 1.875 ms; mean: 7.947 ms; p99: 151.3 ms; max: 262.4 ms EFS · 每個 request 嘅 stat 時間 — median: 1.875 ms 1.875 ms EFS · 每個 request 嘅 stat 時間 — mean: 7.947 ms 7.947 ms EFS · 每個 request 嘅 stat 時間 — p99: 151.3 ms 151.3 ms EFS · 每個 request 嘅 stat 時間 — max: 262.4 ms 262.4 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.773 ms; mean: 1.775 ms; p99: 1.849 ms; max: 3.454 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.773 ms 1.773 ms EBS 對照組 · 每個 request 嘅 stat 時間 — mean: 1.775 ms 1.775 ms EBS 對照組 · 每個 request 嘅 stat 時間 — p99: 1.849 ms 1.849 ms EBS 對照組 · 每個 request 嘅 stat 時間 — max: 3.454 ms 3.454 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.627 ms; mean: 1.63 ms; p99: 1.72 ms; max: 2.932 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.627 ms 1.627 ms EBS 對照組 · 每個 request 嘅 stat 時間 — mean: 1.63 ms 1.63 ms EBS 對照組 · 每個 request 嘅 stat 時間 — p99: 1.72 ms 1.72 ms EBS 對照組 · 每個 request 嘅 stat 時間 — max: 2.932 ms 2.932 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.501 ms; mean: 1.533 ms; p99: 1.727 ms; max: 2.763 ms EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.501 ms 1.501 ms EBS 對照組 · 每個 request 嘅 stat 時間 — mean: 1.533 ms 1.533 ms EBS 對照組 · 每個 request 嘅 stat 時間 — p99: 1.727 ms 1.727 ms EBS 對照組 · 每個 request 嘅 stat 時間 — max: 2.763 ms 2.763 ms medianmeanp99max

EFS · 每個 request 嘅 stat 時間 — median: 2.14 ms / mean: 6.942 ms / p99: 164.1 ms / max: 233.3 ms

EFS · 每個 request 嘅 stat 時間 — median: 2.091 ms / mean: 7.221 ms / p99: 216.6 ms / max: 244.1 ms

EFS · 每個 request 嘅 stat 時間 — median: 1.875 ms / mean: 7.947 ms / p99: 151.3 ms / max: 262.4 ms

EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.773 ms / mean: 1.775 ms / p99: 1.849 ms / max: 3.454 ms

EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.627 ms / mean: 1.63 ms / p99: 1.72 ms / max: 2.932 ms

EBS 對照組 · 每個 request 嘅 stat 時間 — median: 1.501 ms / mean: 1.533 ms / p99: 1.727 ms / max: 2.763 ms

Log₁₀ scale · ms · Ordered positions · straight segments join supplied observations

每個 request 加總嘅 stat 時間,由中位數到最大值排成階梯,EFS 同 EBS 對照組各三個 replication,同一個對數刻度。對照組保持平直;EFS 喺尾部離開佢。間距係次序性,唔係時間軸。Run results/E5-formal/20261028T160106Z-dd5fce0。

Source: mikeng-io/aws-wordpress@b52192b
results/E5-formal/20261028T160106Z-dd5fce0

Measurement definitions / Command
EFS · 每個 request 嘅 stat 時間
Per-request summed stat time on EFS plain, delta=1 s, eligible formal-v3 replication 1 in chronological order. Percentile ladder, element order median, mean (amortised), p99, max; every element ns/1e6 (milliseconds) at four significant figures, exact ns in raw. Milliseconds because a request's summed stat time runs from ~1.5 ms to ~260 ms. Source JSON pointers (output order): /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_per_request_ns/median; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_per_request_ns/amortised; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_per_request_ns/p99; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/efs_per_request_ns/max.
EFS · 每個 request 嘅 stat 時間
Per-request summed stat time on EFS plain, delta=1 s, eligible formal-v3 replication 2 in chronological order. Percentile ladder, element order median, mean (amortised), p99, max; every element ns/1e6 (milliseconds) at four significant figures, exact ns in raw. Milliseconds because a request's summed stat time runs from ~1.5 ms to ~260 ms. Source JSON pointers (output order): /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_per_request_ns/median; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_per_request_ns/amortised; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_per_request_ns/p99; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/efs_per_request_ns/max.
EFS · 每個 request 嘅 stat 時間
Per-request summed stat time on EFS plain, delta=1 s, eligible formal-v3 replication 3 in chronological order. Percentile ladder, element order median, mean (amortised), p99, max; every element ns/1e6 (milliseconds) at four significant figures, exact ns in raw. Milliseconds because a request's summed stat time runs from ~1.5 ms to ~260 ms. Source JSON pointers (output order): /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_per_request_ns/median; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_per_request_ns/amortised; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_per_request_ns/p99; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/efs_per_request_ns/max.
EBS 對照組 · 每個 request 嘅 stat 時間
Per-request summed stat time on the matched-EBS control, delta=1 s, eligible formal-v3 replication 1 in chronological order. Percentile ladder, element order median, mean (amortised), p99, max; every element ns/1e6 (milliseconds) at four significant figures, exact ns in raw. Milliseconds because a request's summed stat time runs from ~1.5 ms to ~260 ms. Source JSON pointers (output order): /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/ebs_per_request_ns/median; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/ebs_per_request_ns/amortised; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/ebs_per_request_ns/p99; /predictions/9/per_replication/20261028T103017Z-342d6b4~1rep-1/ebs_per_request_ns/max.
EBS 對照組 · 每個 request 嘅 stat 時間
Per-request summed stat time on the matched-EBS control, delta=1 s, eligible formal-v3 replication 2 in chronological order. Percentile ladder, element order median, mean (amortised), p99, max; every element ns/1e6 (milliseconds) at four significant figures, exact ns in raw. Milliseconds because a request's summed stat time runs from ~1.5 ms to ~260 ms. Source JSON pointers (output order): /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/ebs_per_request_ns/median; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/ebs_per_request_ns/amortised; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/ebs_per_request_ns/p99; /predictions/9/per_replication/20261028T125412Z-f6659e0~1rep-1/ebs_per_request_ns/max.
EBS 對照組 · 每個 request 嘅 stat 時間
Per-request summed stat time on the matched-EBS control, delta=1 s, eligible formal-v3 replication 3 in chronological order. Percentile ladder, element order median, mean (amortised), p99, max; every element ns/1e6 (milliseconds) at four significant figures, exact ns in raw. Milliseconds because a request's summed stat time runs from ~1.5 ms to ~260 ms. Source JSON pointers (output order): /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/ebs_per_request_ns/median; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/ebs_per_request_ns/amortised; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/ebs_per_request_ns/p99; /predictions/9/per_replication/20261028T160106Z-dd5fce0~1rep-1/ebs_per_request_ns/max.
python3 analysis/export_figures.py

第二把尺係喺量度前至少 120 秒,由另一個 mount 度 seed 好啲 file,所以每個量度用嘅 mount 見到佢哋嗰陣都係 cold 嘅。一個 client 每 0.1 秒、每 1 秒或者每 10 秒 stat 每個 file 一次,warm-up 300 秒,量 600 秒。Miss 係用 mountstats 嘅 RPC counter 去分,唔係用 latency threshold,因為 counter 話你知個 call 有冇去到 wire,而 timer 只話你知佢用咗幾耐。不過呢啲 counter 數嘅係 RPC operation,同 file-miss event 唔係同一個量,所以我引用個 rate,唔會換算成 hit percentage。

喺一秒一個 cadence 度,EFS 每個 positive stat 量大約 0.017 個 RPC operation;0.1 秒嘅話係 0.0017,10 秒嘅話係 0.16。所以喺一個現實嘅 cadence 度,wire 上面嘅活動係好少,client 幾乎全部喺 memory 度答。咁樣換到嘅,係 median request 大約 2 ms aggregate stat time,三次 replication 分別 2.14 / 2.09 / 1.88 ms,而 EBS 係 1.77 / 1.63 / 1.50 ms。Mean 係 3.91 倍、4.43 倍、5.19 倍。p99 係 151 到 217 ms,maximum 去到 233 到 262 ms(cadence finding,E5-attribute-cache)。呢啲全部係 EC2 普通 host-mount 嗰條 arm 上面嘅 aggregate stat timing,唔係 end-to-end page latency,亦都唔係 WordPress 嗰條 path。

個形狀之所以睇得清,係因為集中。最慢嗰 5% 嘅 request 揹住咗全部 NFS RPC operation 嘅 82.9%、89.4% 同 82.5%。第一個 replication 嘅 raw per-request timing(我係逐個手睇,唔係經一個 commit 咗嘅 derivation)入面,慢過 20 ms 嘅 request 喺 EFS 上面係一叢一叢咁嚟,每叢相隔 56 到 59 秒,而 EBS control 完全冇。不過呢個檢查只覆蓋一個 replication。佢同 60 秒 window 對得上,亦都同「一齊 compile 嘅 file 一齊到期」對得上,但兩個都仍然係推斷;數據入面冇嘢分辨得出「同步到期」同其他任何一個會令 request 一齊爆嘅原因。

Lustre 係用嚟做對照,睇「完全冇 attribute timer」係咩環境:佢慢 call 嘅佔比喺每個 interval 都停喺 0.157 到 0.335%,而個實驗設嘅上限係 1%;同時 EBS control 係 0.0000 到 0.0243%;Lustre 用嘅係佢 Scratch 2、1200 GiB、version 2.15 嘅 configuration。呢個係 latency 觀察,唔係 lock state 嘅證明。

讀數出錯嘅位就係個 mean。佢量到嘅 aggregate stat time 係真做嘅工,如果你個 model 講嘅係總和,佢係一個企得住嘅 input;但佢描述唔到典型嘅 request(大約 2 ms),亦都描述唔到嗰啲集中嘅停頓(151 到 217 ms)。呢啲 aggregate 等待會唔會變成 fleet 成本,要睇 concurrency、worker occupancy 同你個 latency target——呢個 study 一樣都冇量。

Launch type 決定你個餐牌有咩揀

EFS

0100EFS — replication 1 of 3: 33.5%33.5%EFS — replication 2 of 3: 37.8%37.8%EFS — replication 3 of 3: 30.9%30.9%

33.5% / 37.8% / 30.9%

FSx OpenZFS

0100FSx OpenZFS — replication 1 of 3: 69.6%69.6%FSx OpenZFS — replication 2 of 3: 68.9%68.9%FSx OpenZFS — replication 3 of 3: 67.9%67.9%

69.6% / 68.9% / 67.9%

FSx ONTAP

0100FSx ONTAP — replication 1 of 3: 100%100%FSx ONTAP — replication 2 of 3: 100%100%FSx ONTAP — replication 3 of 3: 100%100%

100% / 100% / 100%

EFS

0100EFS — replication 1 of 3: 33.5%33.5%EFS — replication 2 of 3: 37.8%37.8%EFS — replication 3 of 3: 30.9%30.9%

33.5% / 37.8% / 30.9%

FSx OpenZFS

0100FSx OpenZFS — replication 1 of 3: 69.6%69.6%FSx OpenZFS — replication 2 of 3: 68.9%68.9%FSx OpenZFS — replication 3 of 3: 67.9%67.9%

69.6% / 68.9% / 67.9%

FSx ONTAP

0100FSx ONTAP — replication 1 of 3: 100%100%FSx ONTAP — replication 2 of 3: 100%100%FSx ONTAP — replication 3 of 3: 100%100%

100% / 100% / 100%

Linear scale · percent · Supplied replications

由 client memory 直接回答嘅 stat 佔比,用文中講明、並標明係事後回看定嘅描述性規則:EFS 33.5% / 37.8% / 30.9%,ONTAP 100% / 100% / 100%。建立檔案最快嗰個 tier 睇落最快,因為佢啲檔案仲喺 attribute-cache 窗口入面。Run results/E2/20261015T000959Z-85382cc。

Source: mikeng-io/aws-wordpress@b52192b
results/E2/20261015T000959Z-85382cc

Measurement definitions / Command
EFS
share of the 1,000 stat calls answered in client memory under the descriptive, post-hoc 20 us rule (stat latency < 20 us = answered in client memory), declared after inspecting the run's samples and not pre-registered; every absolute threshold in the 15-30 us sensitivity range yields identical shares, so the value is not tuned. It disagrees with the repo's registered E3-gated k-means clustering rule, which leaves ONTAP and Lustre undefined and is kept as per-rep diagnostics (registered_rule) in the derived file, per replication; derived by analysis/e2_cache_window.py from the pinned FSx run's raw CSVs; the array is the three replications. Source JSON pointers (output order): /tiers/efs/replications/rep-1/answered_local_share; /tiers/efs/replications/rep-2/answered_local_share; /tiers/efs/replications/rep-3/answered_local_share.
FSx OpenZFS
share of the 1,000 stat calls answered in client memory under the descriptive, post-hoc 20 us rule (stat latency < 20 us = answered in client memory), declared after inspecting the run's samples and not pre-registered; every absolute threshold in the 15-30 us sensitivity range yields identical shares, so the value is not tuned. It disagrees with the repo's registered E3-gated k-means clustering rule, which leaves ONTAP and Lustre undefined and is kept as per-rep diagnostics (registered_rule) in the derived file, per replication; derived by analysis/e2_cache_window.py from the pinned FSx run's raw CSVs; the array is the three replications. Source JSON pointers (output order): /tiers/fsx_openzfs/replications/rep-1/answered_local_share; /tiers/fsx_openzfs/replications/rep-2/answered_local_share; /tiers/fsx_openzfs/replications/rep-3/answered_local_share.
FSx ONTAP
share of the 1,000 stat calls answered in client memory under the descriptive, post-hoc 20 us rule (stat latency < 20 us = answered in client memory), declared after inspecting the run's samples and not pre-registered; every absolute threshold in the 15-30 us sensitivity range yields identical shares, so the value is not tuned. It disagrees with the repo's registered E3-gated k-means clustering rule, which leaves ONTAP and Lustre undefined and is kept as per-rep diagnostics (registered_rule) in the derived file, per replication; derived by analysis/e2_cache_window.py from the pinned FSx run's raw CSVs; the array is the three replications. Source JSON pointers (output order): /tiers/fsx_ontap/replications/rep-1/answered_local_share; /tiers/fsx_ontap/replications/rep-2/answered_local_share; /tiers/fsx_ontap/replications/rep-3/answered_local_share.
python3 analysis/export_figures.py

下面個表講緊嘅規則係:你有邊啲 tier 可以用,喺你問任何 performance 問題之前,就已經由你個 launch type 決定咗。

Tier喺 EC2喺 Fargate邊個決定
EFSECS-managed volume,或者 host mount 再 bindECS-managed volume,platform 1.4.0+mount helper、port 2049
EBS gp3ECS volume 或者 root volumeAWS 文件寫住係一個 volume option冇量過嘅 Fargate arm
FSx for Lustrehost mount 再 bind唔 support唔係 ECS volume type
FSx for OpenZFShost mount 再 bind唔 support唔係 ECS volume type
FSx for ONTAPtask-definition volume,native NFS淨係 S3 Access Points,S3 API 唔係 POSIX要 native NFS 就用 EC2
S3 Access PointsS3 API連得到,但唔係 mountobject semantics,唔係 POSIX

上面講嘅 eligibility 係睇文件,唔係量出嚟(ECS data volumes)。FUSE 類 tier 冇咗,係因為平台原因而唔係 test 結果:佢哋要 CAP_SYS_ADMIN,而 Fargate 只准你加一個 Linux capability,就係 CAP_SYS_PTRACE(Fargate task considerations)。呢個 study 入面所有 FUSE 量度都係喺 EC2 做,Fargate 上面一個都冇試過。

TierPinned configuration
EFSElastic throughput、One Zone、General Purpose、transit encryption 開、NFS 4.1
FSx for OpenZFSSingle-AZ、64 GiB、128 MBps provisioned(唔係 64 MBps 個底)、compression NONE
FSx for ONTAPSingle-AZ gen-1、1024 GiB、128 MBps、冇 snapshot、storage efficiency 熄
FSx for LustreScratch 2、1200 GiB、version 2.15、mount 用 -o flock

每一行都係一個 provisioning point,唔係成個 service 嘅 latency 特性。Provisioned throughput 會決定 FSx file server 嘅 in-memory metadata cache 有幾大,所以 OpenZFS 同 ONTAP 啲數有一部分係個 cap 造成;Scratch 2 唔係 Persistent,呢度亦都冇一樣嘢係講 Persistent 嗰種 deployment type。encrypted EFS 嗰行係屬於 per-call campaign;上面嘅 cadence 結果係嚟自普通 host-mount 嗰條 arm。

量咗三個 filesystem,而老實講,個總結比「淘汰賽」窄好多。JuiceFS 做到 104.6 µs stat p50,create 29.2 ms,主要係花咗喺將 file body 寫上 S3;佢本來係登記嚟答「object-backed 嘅 userspace filesystem 喺 metadata 上面可唔可以贏 EFS 一個數量級」,而 7.5 倍快過 EFS cache-miss p50,結果係 inconclusive,唔算贏唔算輸(FUSE results,E2-fuse)。SeaweedFS 做到 20.2 µs stat、2.28 ms create,11 個 conformance check 過晒 11 個,即係喺 EC2 上面,佢係一個真 option,只係呢個 study 冇攞佢對住個 application 試。Mountpoint for S3 喺每個 replication 都係 11 個 single-node check 錯 8 個,包括 rename 唔會覆蓋原本嗰個、mtime 唔會前進,所以佢冇 latency table:semantic 上錯,唔可以就咁撈入 performance ranking 度平均。三個共通點係都唔 eligible 喺 Fargate 用,但呢個係平台排除,唔係對嗰個 software 嘅判決。

Hydration 抽走咗「到期」呢個事件

  • EFS · stat
  • 已 hydrate 嘅本地目錄樹 · stat
110100100010000 EFS · stat — p50: 1388 µs; p95: 1745 µs; p99: 5106 µs EFS · stat — p50: 1388 µs 1388 µs EFS · stat — p95: 1745 µs 1745 µs EFS · stat — p99: 5106 µs 5106 µs 已 hydrate 嘅本地目錄樹 · stat — p50: 3.061 µs; p95: 3.938 µs; p99: 6.023 µs 已 hydrate 嘅本地目錄樹 · stat — p50: 3.061 µs 3.061 µs 已 hydrate 嘅本地目錄樹 · stat — p95: 3.938 µs 3.938 µs 已 hydrate 嘅本地目錄樹 · stat — p99: 6.023 µs 6.023 µs p50p95p99

EFS · stat — p50: 1388 µs / p95: 1745 µs / p99: 5106 µs

已 hydrate 嘅本地目錄樹 · stat — p50: 3.061 µs / p95: 3.938 µs / p99: 6.023 µs

110100100010000 EFS · stat — p50: 1388 µs; p95: 1745 µs; p99: 5106 µs EFS · stat — p50: 1388 µs 1388 µs EFS · stat — p95: 1745 µs 1745 µs EFS · stat — p99: 5106 µs 5106 µs 已 hydrate 嘅本地目錄樹 · stat — p50: 3.061 µs; p95: 3.938 µs; p99: 6.023 µs 已 hydrate 嘅本地目錄樹 · stat — p50: 3.061 µs 3.061 µs 已 hydrate 嘅本地目錄樹 · stat — p95: 3.938 µs 3.938 µs 已 hydrate 嘅本地目錄樹 · stat — p99: 6.023 µs 6.023 µs p50p95p99

EFS · stat — p50: 1388 µs / p95: 1745 µs / p99: 5106 µs

已 hydrate 嘅本地目錄樹 · stat — p50: 3.061 µs / p95: 3.938 µs / p99: 6.023 µs

Log₁₀ scale · us · Ordered positions · straight segments join supplied observations

EFS 同一個喺開工時 hydrate 去 task-local storage 嘅程式碼樹,兩者嘅 stat 延遲,p50 到 p99,同一個對數刻度。hydration 喺開工時要付幾多代價,文中已經講明。間距係次序性。Run results/E4/20261029T091909Z-842d596。

Source: mikeng-io/aws-wordpress@b52192b
results/E4/20261029T091909Z-842d596

Measurement definitions / Command
EFS · stat
EFS stat percentile ladder, pooled across three deployments; element order p50, p95, p99; every element ns/1000 at four significant figures, exact ns in raw. Source JSON pointers (output order): /pooled/efs/stat/p50_ns; /pooled/efs/stat/p95_ns; /pooled/efs/stat/p99_ns.
已 hydrate 嘅本地目錄樹 · stat
Hydrated code-tree stat percentile ladder, pooled across three deployments; element order p50, p95, p99; every element ns/1000 at four significant figures, exact ns in raw. Source JSON pointers (output order): /pooled/hydrated/stat/p50_ns; /pooled/hydrated/stat/p95_ns; /pooled/hydrated/stat/p99_ns.
python3 analysis/export_figures.py

喺 startup 嗰陣,將一個 read-only code tree 由 EFS hydrate 落 task 自己嘅 ephemeral storage,並冇消除 filesystem access。佢消除嘅係嗰條 read path 上面嘅 remote attribute-cache 到期。三個獨立 deployment,Fargate 1.4.0、ARM64、0.5 vCPU 同 1 GiB、20 GiB ephemeral(hydration finding,E4-hydration):

量度項目喺 EFShydrate 之後比例
stat p501.3884 ms3.061 µs0.00220
open_read p502.7109 ms11.667 µs0.00430
Tar 解壓,27,412,480 bytesn/a0.218 / 0.444 / 0.429 秒同一批 byte,差距 2.04 倍
由第一行 log 到 benchmark 開始,兩條 arm 之間n/a0.051 / 0.192 / 0.265 秒有 confound,見下

上面嘅 3.061 µs 同 0.00220,係呢個 experiment 報告嘅 summary 值,係喺 task-local storage 上面逐個 operation 量。每個 replication 都企得住嘅,係佢哋背後嗰條 acceptance threshold:hydrate 之後唔可以差過 EFS stat 成本嘅十分一,同 open_read 成本嘅五分一;逐個 run 嘅結果係圍住個 summary 上上落落,唔係逐個 digit 複製得出。本地啲 distribution 係單峰嘅,stat p99 係 6.023 µs、ENOENT p99 係 22.6 µs,而呢個就係重點:local filesystem 冇 revalidation timer,所以根本冇嘢會到期。

我事前預測 hydration 會令 benchmark 至少遲 0.5 秒先開得工,結果錯,而且係錯喺「平」嗰邊。解壓本身用咗 0.218 到 0.444 秒,而兩條 arm 之間「第一行 log 到 benchmark」嘅 delta 係 0.051 到 0.265 秒,當中仲包含 remote conformance write 同 scheduling noise,所以呢個 delta 我唔會當係 hydration 嘅因果 overhead,亦都唔會當係成個 startup time。

呢個唔係一個量到嘅 request-latency 結果。hydrate 咗嘅 tree 從來冇喺 cadence 嗰把尺嘅負載下面跑過,所以 151 到 217 ms 嗰個停頓有冇消失,仲未有人證明到;兩者之間嘅關連,係由兩個獨立結果推返嚟嘅推斷。而且佢好 specific:一個 read-only code tree,得 27 MB 而唔係一個 production install,冇 write-back adapter、冇 fleet refresh 或者 invalidation 行為、亦都冇 end-to-end latency 量度。佢處理嘅係一條 path 上面嘅到期事件。佢唔會令 mutable 嘅 application state 變成本地,而且佢只係一個有範圍嘅 option,唔係剩低唯一嗰個。

Audit notes:呢個 study 講得出啲乜

Study 問過嘅問題結果
每個 server-backed tier 喺 stat 上面係咪都會慢過 block group 10 倍以上?機械式咁被推翻:表面嘅反例其實係 client-cache hit 而唔係 server lookup,所以呢個推翻本身冇乜資訊量
first touch 係咪貴過 revalidation?喺 EFS 係,2.8 到 4.4 倍;事前登記嘅 1.0 到 3.0 區間兩邊都被推翻——EFS 嗰幾條 arm 喺上面超咗,有一個 OpenZFS replication 喺下面 0.72 倍
一個 request 係咪要畀 per-call 個 ratio?唔係:mean 落喺佢 support bound 之內,而且遠低過推翻門檻(cadence finding,E5-attribute-cache)
hydration 嘅成本係咪低過我預測、benchmark 開工前嗰 0.5 秒下限?唔係;解壓用咗 0.218 到 0.444 秒,所以個預測被推翻,而兩條 arm 之間嘅 delta 太 confounded,唔可以攞嚟代替
hydration 係咪消除咗 request 停頓?未證明到;從來冇喺 cadence 嗰把尺下面跑過

啲預測喺跑之前已經登記好,設計容許嘅地方做三次 replication,每個 service 都只喺一個 provisioning point 量。Wire 上面有冇活動係靠 RPC counter 講,wall-clock latency 永遠唔會用嚟撐「個 call 有冇離開 client」呢類講法。20 µs 嗰條 cache-hit cutoff、得一個 replication 嘅 clustering 檢查、同「同步到期」嘅解釋,喺每一處出現都標明係 post-hoc。冇量過 end-to-end request latency、冇一個 cadence 對得上嘅 Fargate run、冇試過刻意 share host mount,cart 同 checkout traffic 有數但冇做 end-to-end instrumentation。

星期一開工:decision checklist

  1. 喺比較任何 storage tier 之前,先喺一個 warm request 度數清楚你嘅 filesystem 需求。你冇量過嘅需求數字,計唔到 capacity。
  2. 睇清楚你個 launch type 到底 attach 得到啲乜,因為一個對你嚟講唔存在嘅 option,唔係一個慢啲嘅 option。
  3. 先試你個 application 需要嘅 POSIX semantics。rename 同 mtime 就係 object-store mount 死嘅位,而 correctness 出事係排除,唔係排名。
  4. 用你自己嘅 cadence 去量,用 RPC counter 而唔係秒錶,因為「啱啱 create 嘅 file 矩陣」同「request 形狀嘅 sweep」答緊唔同嘅問題。
  5. 你重複引用任何一個數,都要連佢個 provisioning point 一齊講。Single-AZ OpenZFS 128 MBps 同 Single-AZ gen-1 ONTAP 1024 GiB 係 configuration,唔係成個 service。
  6. 如果你需求入面「code」嗰部分係 read-only,可以考慮 hydrate 佢落 task-local storage,並且量埋解壓成本同逐個 operation 嘅結果。如果唔係 read-only,呢個 option 唔關你事。

對呢個 workload 嚟講,storage tier 決定條尾個形狀,同佢決定個平均一樣咁多;而真正抽走「到期」呢個事件嘅干預,係改個 code 由邊度讀。Mean 嘅 penalty 仲喺度,任何計總 stat 等待嘅數都要計埋佢;但佢從來都冇描述過一個獨立 request,而用戶記得嘅 request,永遠都係 p99 嗰啲。