我试图用一个非常简单的例子来测试parMap vs map： import Control.Parallel.Strategiesimport Criterion.Mainsq x = x^2a = whnf sum $map sq [1..1000000]b = whnf sum $parMap rseq sq [1..1000000]main = defaultMain [ bench "1" a, be
import Control.Parallel.Strategies
import Criterion.Main
sq x = x^2
a = whnf sum $map sq [1..1000000]
b = whnf sum $parMap rseq sq [1..1000000]
main = defaultMain [
bench "1" a,
bench "2" b
]
我的结果似乎表明parMap没有加速,我想知道为什么会这样？
benchmarking 1
Warning: Couldn't open /dev/urandom
Warning: using system clock for seed instead (quality will be lower)
time 177.7 ms (165.5 ms .. 186.1 ms)
0.997 R² (0.992 R² .. 1.000 R²)
mean 185.1 ms (179.9 ms .. 194.1 ms)
std dev 8.265 ms (602.3 us .. 10.57 ms)
variance introduced by outliers: 14% (moderately inflated)
benchmarking 2
time 182.7 ms (165.4 ms .. 199.5 ms)
0.993 R² (0.976 R² .. 1.000 R²)
mean 189.4 ms (181.1 ms .. 195.3 ms)
std dev 8.242 ms (5.896 ms .. 10.16 ms)
variance introduced by outliers: 14% (moderately inflated)
问题是parMap激发了每个列表元素的并行计算.根据您的评论,您似乎根本不需要使用
parListChunk策略.
因此parMap具有很高的开销,因此每个spark简单地对一个数字进行平方的事实意味着它的成本被这个开销所淹没.