for x in nparr:
if x == 0:
something something
as that uses a lot more time than doing this
for x in nparr.tolist():
if x == 0:
something something
This is because a for loop iterating over a numpy array does not result in a sequence of Python constants but in a sequence of numpy scalars which would result in comparing a numpy array to a constant. Converting the array into a list first before the for loop will then result in a sequence of constants.
Here is some profiling I've done using cProfile to check different ways to do an 'if' on a numpy array element:
import cProfile
import numpy as np
runs = 1000000
print('Comparing numpy to numpy')
x = np.array(1.0, np.float32)
y = np.array(1.0, np.float32)
cProfile.run('''
for _ in range(runs):
if x == y:
pass
''')
print()
print('Comparing numpy to constant')
x = np.array(1.0, np.float32)
cProfile.run('''
for _ in range(runs):
if x == 1.0:
pass
''')
print()
print('Comparing constant to constant')
x = 1.0
cProfile.run('''
for _ in range(runs):
if x == 1.0:
pass
''')
print()
print('Comparing numpy.tolist() to constant')
x = np.array(1.0, np.float32)
cProfile.run('''
for _ in range(runs):
if x.tolist() == 1.0:
pass
''')
print()
print('Comparing numpy to numpy.array(constant)')
x = np.array(1.0, np.float32)
cProfile.run('''
for _ in range(runs):
if x == np.array(1.0, np.float32):
pass
''')
print()
print('Comparing numpy.tolist() to numpy.tolist()')
x = np.array(1.0, np.float32)
y = np.array(1.0, np.float32)
cProfile.run('''
for _ in range(runs):
if x.tolist() == y.tolist():
pass
''')
print()
Here are the results in order of speed:
| Comparing constant to constant: | 0.088 seconds |
|---|---|
| Comparing numpy.tolist() to constant: | 0.288 seconds |
| Comparing numpy.tolist() to numpy.tolist(): | 0.508 seconds |
| Comparing numpy to numpy: | 0.684 seconds |
| Comparing numpy to constant: | 1.192 seconds |
| Comparing numpy to numpy.array(constant): | 1.203 seconds |
It turns out that it is always faster to first convert your numpy scalars into constants via .tolist() than to do anything with them as numpy scalars.
No comments:
Post a Comment