You don't need to make this judgement; it's fixed by the colorspace you're working in.

First, figure out what colorspace the processing needs to happen in. Usually this is linear RGB.

Then, figure out what OETF and EOTF your input/output format use. This will be something like PQ or HLG. This will exactly specify the meaning of each integer value.

This fixes the choice of representation and conversion.