Trac: Speed up simulations in Python implementation

[MichaelClerx]

See https://chaste.cs.ox.ac.uk/trac/ticket/2459

• Try using Cython to speed up simulations
  • Also see if we can do any better about avoiding creating lots of numpy views of NVectors etc.
• Investigate better use of numba?
  • Bug report at ​Jitting this code takes an unexpectedly long time numba/numba#318
  • Running TestIcalProto.py with numba has just completed in 7125.190s, of which about 6685s (110mins) was compiling the model.
• Hotspots identified by profiling TestIcalProto etc (and revision addressed in):
  • Cython GetOutputs is mostly running Python arithmetic not C - more cdef's (r20656, r20710)
    • Also, don't create an environment with wrapped values that then get unwrapped - just return the dict of unwrapped values directly (r20662:20663)
    • Avoid object creation (r23437)
    • Return a list, not dict (r23441:23442)
  • AbstractSimulation.LoopBodyStartHook - mostly applying modifiers
    • Look at whether the SetValue calculation can be optimised
      • When evaluating children, don't wrap then unwrap
      • Look at optimising with ​cython.inline or similar (r20700, r20709)
      • Have differentiated compile results for array and single value contexts (started in r20709)
    • Optimise Environment class
      • r20663 provides it with a partial dict interface which makes iteration slightly faster
  • AbstractSimulation.AddIterationOutputs (r23436)
    • Use iterator comprehensions rather than creating a temporary list (r20657)
    • Investigate whether ranges can be done differently to speed up creating a tuple of range indices (r23443)
    • Might be worth investigating whether we can write directly to the correct portion of the final outputs arrays, rather than writing to arrays within the model then copying, which takes the majority of the time in this method now it seems.
  • TimecourseSimulation.InternalRun - cache attribute lookup within loop (r20657)
  • Changing model variables via the wrapper environment
    • Make it easier to get the unwrapped version of a value (r20657)
    • Don't wrap then immediately unwrap whenever possible (r20700)
    • Optimise the model wrapper dict subclasses (r20663)
  • Only parse the protocol once - use the first parse to generate XML version for Pycml too (partly in r20697)
  • Special case evaluation of the Join built-in

[MichaelClerx]

Extensive discussion in original ticket