Processing varints with the single_varint() function

The single_varint function finds the first varint within the supplied data and uses an index to keep track of its current position. When it finds the varint, it returns the value along with the index. This tells the calling function how many bytes the varint was and is used to update its own index:

243 def single_varint(data, index=0):
244 """
245 The single_varint function processes a Varint and returns the
246 length of that Varint.
247 :param data: The data containing the Varint (maximum of 9
248 bytes in length as that is the maximum size of a Varint).
249 :param index: The current index within the data.
250 :return: varint, the processed varint value,
251 and index which is used to identify how long the Varint was.
252 """

For this script, we've made a simplifying assumption that varints will never be greater than 2 bytes. This is a simplifying assumption and won't be appropriate in all situations. This leaves two possible scenarios:

Based on the outcome, one of the following two things will happen. If the byte is greater than or equal to 128, the varint is 2 bytes long. Otherwise, it's only 1 byte in length. On lineĀ 256, we use the ord() function to convert the value of the byte into an integer:

254     # If the decimal value is => 128 -- then first bit is set and
255 # need to process next byte.
256 if ord(data[index:index+1]) >= 128:
257 # Check if there is a three or more byte varint
258 if ord(data[index + 1: index + 2]) >= 128:
259 raise ValueError

If the value is greater than 128, we know that the second byte is also required and must apply the following generic formula, where x is the first byte and y is the second byte:

Varint = ((x - 128) * 128) + y

We return this value after incrementing the index by 2:

260         varint = (ord(data[index:index+1]) - 128) * 128 + ord(
261 data[index + 1: index + 2])
262 index += 2
263 return varint, index

If the first byte is less than 128, all we must do is return the byte's integer value and increment the index by 1:

265     # If the decimal value is < 128 -- then first bit isn't set 
266 # and is the only byte of the Varint.
267 else:
268 varint = ord(data[index:index+1])
269 index += 1
270 return varint, index