Event Filtering
Because every viewer can send events, we could end up with a massive number of events that the game process can’t handle efficiently.
To solve this problem, all events sent go through a set of MapReduce filters to transform a potentially huge number of events into a much more reasonable count.
We create these filters using JSON files and need to load them into the events
service to function properly. In our samples, we have an events.json
file
which includes the map and reduce that we load into the config
folder.
Map example
The map function takes key:value
pairs as input and transforms each into
another key:value
pair when they meet certain conditions. We refer to this as
the mapping step.
Genvid defines both the map and the transformation using a JSON file.
{
"version": "1.7.0",
"event": {
"game": {
"maps": [
{
"id": "changeColor",
"source": "userinput",
"where": {"key": ["changeColor", "<name>"], "name": "<color>", "type": "string"},
"key": ["changeColor", "<name>", "<color>"], "value": 1
},
{
"id": "cheer",
"source": "userinput",
"where": {"key": ["cheer"], "name": "<name>", "type": "string"},
"key": ["cheer", "<name>"], "value": 1
}
]
}
}
}
In this example, we have the changeColor
event as indicated in the event generation page example
which is the first example that we use in the tutorial sample. When a key matching ["changeColor", "<name>"]
enters the system,
it produces a new key:value
pair.
The new key becomes ["changeColor", "<name>", "<color>"]
, where:
"<name>"
corresponds to any name sent by the event as the second key.
"<color>"
corresponds to the string assigned as the value of the key.
The mapping step finishes by assigning the value indicated in the map as the new value for this pair; in this case the value is “1”.
For example, assume the following events come in from multiple viewers:
[
{"key": ["changeColor", "Aramis"], "value": "red"},
{"key": ["changeColor", "Porthos"], "value": "green"},
{"key": ["changeColor", "Porthos"], "value": "green"},
{"key": ["changeColor", "Athos"], "value": "blue"}
]
The map changeColor
transforms them into the following:
[
{"key": ["changeColor", "Aramis", "red"], "value": 1},
{"key": ["changeColor", "Porthos", "green"], "value": 1},
{"key": ["changeColor", "Porthos", "green"], "value": 1},
{"key": ["changeColor", "Athos", "blue"], "value": 1}
]
In the case of the second example where the cheer event is sent from multiple viewers:
[
{"key": "cheer", "value": "Aramis"},
{"key": "cheer", "value": "Porthos"},
{"key": "cheer", "value": "Porthos"},
{"key": "cheer", "value": "Athos"}
]
The map cheer
transforms them into the following:
[
{"key": ["cheer", "Aramis"], "value": 1},
{"key": ["cheer", "Porthos"], "value": 1},
{"key": ["cheer", "Porthos"], "value": 1},
{"key": ["cheer", "Athos"], "value": 1}
]
Once the data is transformed, we apply the reduce operation.
Reduce example
The reduce operation takes key:value
pairs as input and transforms them into
another set of key:value
pairs, but merged together to have fewer of them.
Like the mapping step, Genvid uses JSON for the reduction step.
{
"version": "1.7.0",
"event": {
"game": {
"reductions": [
{
"id": "changeColor",
"where": {"key": ["changeColor", "<name>", "<color>"]},
"key": ["<name>", "<color>"],
"value": ["$count"],
"period": 250
},
{
"id": "cheer",
"where": {"key": ["cheer", "<name>"]},
"key": ["<name>"],
"value": ["$sum"],
"period": 250
}
]
}
}
}
In this example, we have the changeColor
event as indicated earlier as our first sample that we use in the
tutorial sample. During the reduction step, the operation looks for any key starting with "changeColor"
while ignoring the second and third parameter content (since they can be any value).
The operation merges the matching key:value
together and creates a new value
containing the min, max, count and sum of all the values combined together.
For example, we receive the following results from the map function:
[
{"key": ["changeColor", "Aramis", "red"], "value": 1 },
{"key": ["changeColor", "Porthos", "green"], "value": 1 },
{"key": ["changeColor", "Porthos", "green"], "value": 1 },
{"key": ["changeColor", "Athos", "blue"], "value": 1 }
]
The reduce function merges them into the following:
[
{"key": ["Aramis", "red"], "value": {"min": 1, "max": 1, "count": 1, "sum": 1}},
{"key": ["Porthos", "green"], "value": {"min": 1, "max": 1, "count": 2, "sum": 2}},
{"key": ["Athos", "blue"], "value": {"min": 1, "max": 1, "count": 1, "sum": 1}}
]
For our second example, the map would look like this:
[
{"key": ["cheer", "Aramis"], "value": 1},
{"key": ["cheer", "Porthos"], "value": 1},
{"key": ["cheer", "Porthos"], "value": 1},
{"key": ["cheer", "Athos"], "value": 1}
]
And the reduce would transform it into this:
[
{"key": "Aramis", "value": {"min": 1, "max": 1, "count": 1, "sum": 1}},
{"key": "Porthos", "value": {"min": 1, "max": 1, "count": 2, "sum": 2}},
{"key": "Athos", "value": {"min": 1, "max": 1, "count": 1, "sum": 1}}
]
Important
Due to some past bug, the current values always contains all operations, even if they weren’t requested in the configuration. Future version could fix it.