Series processors

---

• • Series, MutSeries, mut_series() and @func
  • SMA, sum and sliding window technique
  • Series behavior during history vs runtime
  • mut_series() function description

Due to the real-time nature of a candle chart, most of tech analysis indicators work according to principles of a sliding window technique. Indie has a Series and MutSeries data types, mut_series() helper function and @func decorator which help to create algorithms that use this technique. We call such algorithms series processors.

Series, MutSeries, mut_series() and @func

Series is a data type that represents series of numeric values like prices of some instrument. Every element of a Series object stores a value at time of some bar on a chart. Series objects have an interface which gives access to their values with square brackets operator. For example, series of close prices of some instrument is ctx.close and current (the last) close price can be accessed with expression ctx.close[0]. The close price at the previous candle is ctx.close[1] and so on.

Series objects like ctx.open, ctx.high, ctx.low, ctx.close and ctx.volume are built-in and read-only. But Indie lets you create your own series objects, put some values that your algorithm calculates into them and use them in the further parts of your indicator code. This is exactly what MutSeries type and mut_series function are intended to be used for.

Function mut_series wraps any primitive number value and returns an object of MutSeries type with that value written into the [0] (the most recent element). This provides access to previous values of that value in the future Indie invocations. In other words, mut_series function is what you need if you want to persist some value over bars.

Functions decorated with @func are invoked on every candle of chart instrument. But mut_series function does not create a new MutSeries object at the place of it's invocation. And this is very different from how normal Python functions behave. That is why mut_series is considered to be a 'syntactic sugar', one of those Indie 'magic' intended to make your indicators code easier to write. Instead, mut_series function creates a MutSeries object just once, puts it somewhere in the @func's state and returns a reference to that object later on.

Because of that mut_series function can work only in a combo with @func decorator. In other words it is allowed to call mut_series function only from another function which is decorated with @func decorator (or @ctx_func or Indie main function).

SMA, sum and sliding window technique

Now it is a good time to look at some of the series processors in action. Let us take some very basic indicator like for example Simple moving average (SMA) and see how it works under the hood. To calculate SMA with length=4 of some price series, we have to divide every element of a corresponding sum (with the same length=4) by it's length. That is why, first we have to look in detail how sum series processor could be implemented.

The sum of length=4 could be calculated very simple. First, the value of sum_0 (where 0 is the bar index) is undefined, because at that point there is only one price is available in the input series. Same with the sum_1 and sum_2. But the value of sum_3 equals to price_0 + price_1 + price_2 + price_3 which is 5 + 2 + 1 + 2 = 10. So far so good and the next sum value, sum_4, equals to price_1 + price_2 + price_3 + price_4 which is 2 + 1 + 2 + 4 = 9 correspondingly. Finally, in the same manner we calculate sum_5 as price_2 + price_3 + price_4 + price_5 which is 1 + 2 + 4 + 6 = 13. And so on, this is illustrated on the diagram:

Calculation of a sum in a straighforward (but non-effective by time) way:

ctx.bar_index:      0,    1,    2,    3,    4,    5, ...
price:              5,    2,    1,    2,    4,    6, ...
                    _,  ...
                    _,    _,  ...
                    _,    _,    _,   ...
                    _,    _,    _,    10, ...
                    _,    _,    _,    10,   9,  ...
sum(length=4):      _,    _,    _,    10,   9,   13, ...

Here is Indie implementation of this algorithm:

# indie:lang_version = 2
from math import nan
from indie import func, Series

@func
def sum(price: Series, length: int) -> Series:
    result = 0.0
    for i in range(length):
      result += price[i]
    return series(result)

Reader may notice that we have to sum the same price values more than once. We may calculate the next sum values more efficiently using the sum value calculated on a previous step. For example sum_4 = sum_3 - price_0 + price_4, sum_5 = sum_4 - price_1 + price_5 and so on. This other way of calculation is exactly the mentioned before sliding window technique. It turns out that with larger lengths the second way is way more efficient in terms of time. And here is the Indie implementation of it:

# indie:lang_version = 2
from math import nan
from indie import func, Series, series

@func
def sum(price: Series, length: int) -> Series:
    s = series(init=0)
    s[0] += price[0]
    result: float
    if len(price) > length:
        s[0] -= price[length]
        result = s[0]
    else:
        result = nan
    return series(result)

NOTE: That if you need a sum or SMA you don't have to implement them by yourself. Simply import them from the standard library like this: from indie.algorithm import sum, sma.

Now, after we have a sum series processor, it is trivial to write SMA as sma series processor too:

@func
def sma(price: Series, length: int) -> Series:
    s = sum(price, length)
    return series(s[0] / length)

As you can see, it is very easy to call one series processor from another. It is very natural to connect them in a sort of a chain. That is why all the functions decorated with @func return Seriesresult.

Finally here is the main() function of our indicator that plots it on chart:

@indicator('SMA Example', overlay_main_pane=True)
def main(ctx):
    result = sma(ctx.close, 12)
    return result[0]

Series behavior during history vs runtime

Suppose we have an indicator that counts green bars of an instrument on a chart:

# indie:lang_version = 2
from indie import indicator, series

@indicator('Green bars counter')
def main(ctx):
    green_count = series(init=0)
    if ctx.close[0] > ctx.open[0]:
        green_count[0] += 1
    return green_count[0]

Bar coloring traditionally works this way: if closing price of a bar is greater than it's opening price, then the bar is green, otherwise it is red. In the Indie code we have a Series green_count object which accumulates the desired count. This count is returned from main which plots a blue line on a chart. On Figure 1 we see how blue line raises up by 1 on every green bar and stays flat if bar is red. First let us take a close look at how this Series object behaves during indicator calculation on a historical bars of some imaginary instrument. Close and open prices of our instrument could be for example:

ctx.time:      t0, t1, t2, t3, t4, t5
ctx.open:       9, 18,  8,  7,  8, 15
ctx.close:     12,  7, 14, 13, 12,  5

History prices ctx.open and ctx.close will be passed to the indicator for calculation one by one from left to right. Indicator will calculate, at time t0, like this "12 is greater than 9, so we add 1 to green_count". On the next calculation, at time t1, it will be "7 is not greater than 18, so do nothing. Value of green_count is stil 1". In the end, green_count series will contain these values:

ctx.time:      t0, t1, t2, t3, t4, t5
green_count:    1,  1,  2,  3,  4,  4

Thus, during the calculation of indicator on historical data everything is simple, predictable and very natural.

Let us suppose that starting from time t6 the realtime begins:

ctx.time:          t6, t6, t6, t7, t7, t7, t7, t8, t8, t9, ...
ctx.open:           6,  6,  6, 11, 11, 11, 11,  7,  7, ...
ctx.close:         10,  2, 12, 14, 12, 16,  9,  5, 12, ...

Our indicator will behave like this:

ctx.time:          t6, t6, t6, t7, t7, t7, t7, t8, t8, t9, ...
green_count:        5,  4,  5,  6,  6,  6,  5,  5,  6, ...
last bar update:            ^               ^       ^

Moments of the final bar update are marked with ^s. They are important, because values calculated in these moments are stored in the end in the green_count series, overwriting all values written there before at the time of the same bar.

Bar usually updates multiple times at realtime. Every realtime update of a bar triggers a calculation of indicator. Right before every such calculation the state of the green_count[0] series element resets to it's previous value (i.e. green_count[1]) - the value it had at the time of previous bar. That is why during the three updates of bar starting at t6, corresponding element of green_count series was set to 5 at first, then reverted back to 4, then finalized at value 5.

If nothing is written in the green_count[0] series element and bar finalizes then this value is carried further to the next series element without change.

mut_series() function description

mut_series(
  reset: indie.Optional[float] = None,
  init: indie.Optional[float] = None,
  size: indie.Optional[int] = None,
  default: indie.Optional[float] = None
) -> indie.MutSeries

Function mut_series creates MutSeries objects which are containers for calculated values in Indie code. The main feature of MutSeries container is ability to store (or 'remember') historical values of some variable, those values could be later accessed with the square brackets operator. At the moment mut_series() function support only float type as MutSeries element.

Parameters:

• reset - is a value that is written into the most recent element of the MutSeries every time mut_series function is executed. The effect of this parameter is the same as:

# this:
s = mut_series(reset_val)

# is the same as:
s = mut_series()
s[0] = reset_val

• init - is a value that is written into the most recent element of the Series object only once after the Series object was created. Or, in other words, it is a value that is written only at the time of the first series funcion call.

# this:
s = mut_series(init=init_val)

# is the same as:
s = mut_series()
if ctx.bar_index == 0:
    s[0] = init_val

• size - is number of elements that MutSeries object should persist in memory. Default and possible minimum size is 2, which means that series of size=2 persists in memory value at the time of the last candle and value at the time of previous candle. Size is automatically expanded during history calculation of an indicator. For example:

# This series allows us to get access to s[0], s[1] and s[2] elements from now
s = series(0.0, size=3)

# We can read previous value which is out of size=3, for example s[5]:
s[0] = s[5] + 1

The size of s was expanded from 3 to 6. It is allowed to get access to s[0], s[1], s[2], s[3] and s[4], s[5] elements after that. It may seem to have not very much sense at first. We set some size, but we are allowed to expand it later. Well, it is allowed to expand the size only during the calculation of indicator history. If the algorithm by some reason calculates lookback offset for some series at runtime, depending on prices for example like this:

max_size = 100
offset = min(math.round(ctx.close[0]), max_size)
s = series(init=0.0, size=max_size)
a = s[offset] # calc something with it...

We cannot predict values of ctx.close series. They could be less than max_size = 100 at first but then during realtime, they could rise higher. That is why we should always limit the max_offset with min and reserve the s's size with size parameter. Otherwise the indicator would fail.

NOTE: MutSeries objects also have a method request_size(new_size: int) which can be used to expand the size explicitly.

• default - is a value that is returned in certain cases when offset is out of bounds of MutSeries object. The certain cases here are only those which happen before all the internal buffers are trimmed (this usually happens right after calculation of history).

mut_series function has a 'get or create' semantics. It means that only one MutSeries object is created per one mut_series very first function call. All subsequent calls of mut_series in exactly the same position in Indie script simply return MutSeries objects created previously. For example:

@indicator('Example')
def main(ctx):
    s1 = mut_series(0.0)
    s2 = mut_series(42.0)
    # ...

In this code only two MutSeries objects are created: s1 and s2. The main function is called multiple times - per every historical candle and per every realtime update of the last candle - thus multiple times mut_series functions are called too. But only when ctx.bar_index == 0 the s1 and s2 objects are created. During all the subsequent calls, s1 and s2 only receive references to existing MutSeries objects.

Please note that MutSeries objects created by mut_series function give read access to any (well, down to some limit) previous values and read-write access only to the last value of the series object.