This was a two-fold problem. The first of which involved the overwriting
of members during a GUILD_SYNC. Since we are requesting a chunk anyway,
we have all the member references we need without actually chunking. The
second problem came from the order. We were doing a GUILD_SYNC and then
doing the chunking requests. We should do the GUILD_SYNC after the
chunking in order to just update presences.
This should hopefully cover all cases where members are added.
There was a bug where an array of chunks received would get entirely
processed if only a single chunk was received. This was fixed by
explicitly bailing early if we're requesting for chunks.
When we don't request offline members, members that are going from
offline to online in the server will become available via the
PRESENCE_UPDATE event. This should be transparent to the user since
technically the member exists in the server and thus on_member_update
is used instead of on_member_join.
Replaced server member lists, channel lists, and private channel lists
with dicts. This allows O(1) lookups and removes (previously it would be
an O(N) operation to lookup or remove). I did pretty extensive testing
and benchmarking to compare the performance of using lists vs using
dicts. Iterating through lists to find an item is only faster in the
average case for extremely small lists (less than 3 items). For 100
items, using a dict is about 10 times faster on average (and about 100
times faster for 1000 items). The overhead in dicts is in memory usage
and initial creation time. Creating and populating a dict is about 2 to
3 times slower than creating and appending items to a list. However this
cost is still tiny. For 1000 items this equates to about a 70
microsecond difference (on an i7 CPU) for populating the entire dict.
The memory overhead for a dict (compared to a list) is about 25-60 KB
per 1000 items (can vary depending on dict resizing).
Originally I wanted to use OrderedDicts to presereve order, but in my
testing OrderedDicts have about 6x the memory overhead compared to
normal dicts.
Previously we created a copy of the object and did some strange
iteration over the data and set the attributes that we thought
were valid. This worked back then in v0.1.0 of the library when
it was written, but it no longer works nowadays when we want to
be as future proof as possible.
Rapptz
Khazhismel
Zeta
Steven Berler