Python从门到精通(四):性能监控

经过上述几章基础基本全覆盖了，可以写一些简单的程序。本章比较简单，就介绍下如何测试Python程序的性能。但这些方式一般适用于开发阶段，因为线上的系统往往比较复杂。

一、CPU

1.1、用unix函数测试

(venv) MacBook:chapter14 liudong$ time python profile_test.py
__main__.countdown time spend: 0.446525065s
counting time spend: 0.671478864s
math.sqrt time spend: 0.063568925s
sqrt time spend: 0.043364856000000174s
math.sqrt time spend: 0.6419396199999998s
sqrt time spend: 0.42504095399999997s

real 0m2.337s
user 0m2.318s
sys 0m0.016s

1.2、使用cProfile模块

需要先安装 pip install cprofiler

(venv) MacBook:chapter14 liudong$ python -m cProfile profile_test.py
__main__.countdown time spend: 0.450556833s
counting time spend: 0.726764442s
math.sqrt time spend: 0.18848818400000011s
sqrt time spend: 0.15792459199999986s
math.sqrt time spend: 1.8039463000000002s
sqrt time spend: 1.5490455310000004s
22000655 function calls (22000645 primitive calls) in 3.597 seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
3/2 0.000 0.000 0.002 0.001 <frozen importlib._bootstrap>:1002(_find_and_load)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:112(release)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:152(__init__)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:156(__enter__)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:160(__exit__)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:166(_get_module_lock)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:185(cb)
5/3 0.000 0.000 0.001 0.000 <frozen importlib._bootstrap>:220(_call_with_frames_removed)
37 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:231(_verbose_message)
1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:241(_requires_builtin_wrapper)
1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:35(_new_module)
3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:351(__init__)

ncalls：相应代码/函数被执行的次数
tottime: 相应代码/函数被执行的总执行时间，不包括命令调用等
percall：tottime/ncalls
cumtime: 相应代码/函数被执行的总执行时间,包括命令调用等
percall：cumtime/ncallsdef fib_seq(n):
res = []
if n > 0:
res.extend(fib_seq(n-1))
res.append(fib(n))
return res

import cProfile
cProfile.run('fib_seq(30)')

1.3、使用line_profiler模块

需要安装pip install line_profiler

(venv) MacBook:chapter16 liudong$ kernprof -l -v fib_test_3.py
Wrote profile results to fib_test_3.py.lprof
Timer unit: 1e-06 s

Total time: 6.80702 s
File: fib_test_3.py
Function: fib at line 1

Line # Hits Time Per Hit % Time Line Contents
==============================================================
1 @profile
2 def fib(n):
3 7049123 1839146.0 0.3 27.0 if n == 0:
4 1346269 298856.0 0.2 4.4 return 0
5 5702854 1450236.0 0.3 21.3 elif n == 1:
6 2178308 486185.0 0.2 7.1 return 1
7 else:
8 3524546 2732599.0 0.8 40.1 return fib(n-1) + fib(n-2)@profile
def fib_seq(n):
res = []
if n > 0:
res.extend(fib_seq(n-1))
res.append(fib(n))
return res


if __name__ == "__main__":
fib_seq(30)

1.4、自定义工具函数

import time
from functools import wraps

def time_use(func):
"""
Decorator that reports the execution time.
:param func:
:return:
"""
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f'func name is: {func.__name__}, time use: {end - start} s')
return result
return wrapper

1.5、自定义性能函数

import time
from functools import wraps

def time_this(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
r = func(*args, **kwargs)
end = time.perf_counter()
print(f'{func.__module__}.{func.__name__} time spend: {end - start}s')
return r
return wrapper

#装饰器
@time_this
def countdown(n):
while n > 0:
n -= 1

countdown(10000000)


from contextlib import contextmanager

@contextmanager
def time_block(label):
start = time.perf_counter()
try:
yield
finally:
end = time.perf_counter()
print(f'{label} time spend: {end - start}s')


with time_block('counting'):
n = 10000000
while n > 0:
n -= 1


from timeit import timeit
print(f"math.sqrt time spend: {timeit('math.sqrt(2)', 'import math')}s")
print(f"sqrt time spend: {timeit('sqrt(2)', 'from math import sqrt')}s")


print(f"math.sqrt time spend: {timeit('math.sqrt(2)', 'import math', number=10000000)}s")
print(f"sqrt time spend: {timeit('sqrt(2)', 'from math import sqrt', number=10000000)}s")


from functools import wraps
def time_this(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.process_time()
r = func(*args, **kwargs)
end = time.process_time()
print(f'{func.__module__}.{func.__name__} : {end - start}')
return r
return wrapper/Users/liudong/PycharmProjects/pythonProject/venv/bin/python /Users/liudong/PycharmProjects/pythonProject/app/chapter14/profile_test.py
__main__.countdown time spend: 0.466810053s
counting time spend: 0.911959559s
math.sqrt time spend: 0.0725984340000001s
sqrt time spend: 0.04869681800000003s
math.sqrt time spend: 0.6943450900000001s
sqrt time spend: 0.48345576300000026s

进程已结束，退出代码为 0

二、内存

2.1、内存使用memory_profiler模块

需要安装memory_profiler

(venv) MacBook:chapter16 liudong$ python -m memory_profiler mem_pro_exp.py
Filename: mem_pro_exp.py

Line # Mem usage Increment Occurrences Line Contents
=============================================================
1 15.805 MiB 15.805 MiB 1 @profile
2 def my_func():
3 23.438 MiB 7.633 MiB 1 a = [1] * (10 ** 6)
4 176.031 MiB 152.594 MiB 1 b = [2] * (2 * 10 ** 7)
5 176.031 MiB 0.000 MiB 1 del b
6 176.031 MiB 0.000 MiB 1 return a@profile
def my_func():
a = [1] * (10 ** 6)
b = [2] * (2 * 10 ** 7)
del b
return a


if __name__ == '__main__':
my_func()