GW2's Mumble Link "context" data format

GW2's Mumble Link "context" data format

in API Development

Posted by: zeeZ.5713

zeeZ.5713

We know Mumble Link uses this data format:

Data format Description
uint32_t uiVersion
uint32_t uiTick
float[ 3 ] fAvatarPosition
float[ 3 ] fAvatarFront
float[ 3 ] fAvatarTop
wchar_t[ 256 ] name
float[ 3 ] fCameraPosition
float[ 3 ] fCameraFront
float[ 3 ] fCameraTop
wchar_t[ 256 ] identity / character name
uint32_t context_len
byte[ 256 ] context
wchar_t[ 2048 ] description

But what about the stuff inside context? GW2 writes context_len as 48 bytes, but what’s inside?

Edit: See https://github.com/arenanet/api-cdi/blob/master/mumble.md

Attachments:

(edited by zeeZ.5713)

GW2's Mumble Link "context" data format

in API Development

Posted by: Nithanim.5243

Nithanim.5243

That is what I found out while experimenting:
byte[ ] context = new byte[ 48 ];

context[0] //could possibly represent the region - 1 for American, 2 for European server
context[4] //could match the server-id
context[28] //map-id (for WvWvW: 94=Red; 95=Green; 96=Blue)

world_id = byte[0]*1000+context[4]

 

edit:
Assumptions/suggestions/some conclusion after a short period of further testing:

context[6]:
overflow-server? it appears that 146=overflow; 159=normal

context[40]:

unknown | note: equals 128 in WvW (needs review)

context[7]:

unknown | note: equals 37 in overflow, 224 in normal (for me at least; tested in lion’s arch); could somewhat be the id of the actual server working in background (needs review)

context[36]:

unknown | note: equals 158 when on miller-home-servers (my home) and 222 on gunnar (as guest) (if not in WvW or overflow) (tested in: gendarran, lion’s arch, bloodtide); equals 5 in WvW; 4 in overflow (needs review)

edit2
(hopefully) fixed: confusion

(edited by Nithanim.5243)

GW2's Mumble Link "context" data format

in API Development

Posted by: Khisanth.2948

Khisanth.2948

Shouldn’t one of these be the layer?

GW2's Mumble Link "context" data format

in API Development

Posted by: olih.6925

olih.6925

byte[4].byte[5].byte[6].byte[7] == ip address of the map/server i think.
/ip in the chat window has the same number.

edit:
byte[44]-byte[47] == uint32 and seems to be the build version of the game (cf https://api.guildwars2.com/v1/build.json)

(edited by olih.6925)

GW2's Mumble Link "context" data format

in API Development

Posted by: zeeZ.5713

zeeZ.5713

byte4.byte5.byte6.byte7 == ip address of the map/server i think.
/ip in the chat window has the same number.

edit:
byte44-byte47 == uint32 and seems to be the build version of the game (cf https://api.guildwars2.com/v1/build.json)

That seems to add up, so I added that above.

Please note that I’m using square brackets to give array sizes, not position :P float[ 3 ] is an array of three floats, not a float at position 3! Thus Nithanim is confusing the heck out of me, even if I assume he means ints and not single bytes.


E: I’ve attached random dumps from Plains of Ashford, Black Citadel, Lion’s Arch, Heart of the Mists, Red, Blue, and Green Borderlands. Didn’t get any overflows.

map_id seems off the charts for WvW.

There seems to be something in byte 32, maybe another int? Everything else is 0 across the board.

(edited by zeeZ.5713)

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

btw. the world_id is the current world_id, which means that you can also see on which overflow server you are

(yes, lookin at you anet!)

€: it doesn’t send the overflow world id, just a garbage value <.<

(edited by smiley.1438)

GW2's Mumble Link "context" data format

in API Development

Posted by: olih.6925

olih.6925

Yeah, sorry for that.
For me, vizunah square : european server
For field not known :

0 – 3 -> uint32 ? value == 2 : so, maybe the region like you said, not really certain without more data
8 – 27 -> always 0 ?
32 – 35 -> uint32 ? 5 in pve, 9 cbe, 11 green map, 10 blue map, 12 red map. ?
40 – 43 -> always 0 ?

:edit: correction, for 32-35

(edited by olih.6925)

GW2's Mumble Link "context" data format

in API Development

Posted by: Nithanim.5243

Nithanim.5243

Thus Nithanim is confusing the heck out of me, even if I assume he means ints and not single bytes.

Sorry for causing confusion…
I do actually mean single bytes. I did only compare when which byte changes.
With byte[ 2 ] i mean the byte at position 2 in “context” as byte-array.

PS: Editing my previous post.

(edited by Nithanim.5243)

GW2's Mumble Link "context" data format

in API Development

Posted by: Cliff Spradlin.3512

Cliff Spradlin.3512

Lead Programmer

Next

You guys did a good job tracking down most of it.

Here’s the whole thing:


// uniquely identifies a map instance for mumble
struct MumbleContext {
    byte serverAddress[28]; // contains sockaddr_in or sockaddr_in6
    unsigned mapId;
    unsigned mapType;
    unsigned shardId;
    unsigned instance;
    unsigned buildId;
};

GW2's Mumble Link "context" data format

in API Development

Posted by: Wothor.4781

Wothor.4781

Thanks for the information
I might use this later on to overcome overflow server disparity for Positional Audio in CrossTalk (for Teamspeak), but I’m sure non-voip related applications will make much more interesting use of those bits.

Might also help for the more-or-less-basic Anti-Spy Protection I’m looking into, but to leverage it to the next level it’d be helpful to have the home world id and possibly guild tag accessible. I’ve also been asked for the class to magically show as icon. While I’m totally fine with waiting for the API to reach those areas, I’ll just put it out there that I could spare you some network traffic by more extensively using idendity (note that game description would be unused yet, too), as mentioned in the Mumble Link description.

For example breaking idendity [255] into char_name[x], char[3]guild_tag, commander_and_whatever_flag[1], class[2], unsigned home_world_id.

Could also be used for options with commander flags, like syncing priority speaker or assigning server/channel groups.

GW2's Mumble Link "context" data format

in API Development

Posted by: Cliff Spradlin.3512

Previous

Cliff Spradlin.3512

Lead Programmer

Next

In the latest patch, the mumble link was enhanced to provide more information about the character in the ‘identity’ field. Hopefully this extra information is useful for API developers.

GW2's Mumble Link "context" data format

in API Development

Posted by: poke.3712

poke.3712

That’s very useful, thanks! Although it would be even better if there was some unique per account information so one could identify when a player switches the character. So either an account name, or some internal ID, or even a hash of anything account identifying would be very good.

@poke // GW2W //

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

That’s very useful, thanks! Although it would be even better if there was some unique per account information so one could identify when a player switches the character. So either an account name, or some internal ID, or even a hash of anything account identifying would be very good.

I had this idea also in conjunction with our location tool but since unique IDs to identify user data are always problematic (ask Google…) i guess we need to rely on user input.

(edited by smiley.1438)

GW2's Mumble Link "context" data format

in API Development

Posted by: poke.3712

poke.3712

Well I don’t want something that could impose a security risk. Something like the account name would work fine. After all, the account name is not really secret; it is visible here in the forums too, and when you add someone to your friendlist using a character name (which is already in the data), then you also see the account name.

@poke // GW2W //

GW2's Mumble Link "context" data format

in API Development

Posted by: Felicja.6891

Felicja.6891

Cliff, thank you for sharing with us the API and continually improving them.

I know this may be a long shot, but…it would be fair to say that attempts to get MumbleLink to work on Mac OS X have been unsuccessful.
Is there any chance that you would be able to confirm with Transgaming that their cider wrapper of GW2 client for mac is actually executing the part of the Gw2 client code that is writing to a shared mapped memory “MumbleLink”?

Also, would it be against the rules to scan the memory of the GW2 client itself and get the necessary values of the coordinates, given the MumbleLink is not working at present on Mac?

(edited by Felicja.6891)

GW2's Mumble Link "context" data format

in API Development

Posted by: Cliff Spradlin.3512

Previous

Cliff Spradlin.3512

Lead Programmer

Next

Cliff, thank you for sharing with us the API and continually improving them.

I know this may be a long shot, but…it would be fair to say that attempts to get MumbleLink to work on Mac OS X have been unsuccessful.
Is there any chance that you would be able to confirm with Transgaming that their cider wrapper of GW2 client for mac is actually executing the part of the Gw2 client code that is writing to a shared mapped memory “MumbleLink”?

Also, would it be against the rules to scan the memory of the GW2 client itself and get the necessary values of the coordinates, given the MumbleLink is not working at present on Mac?

Sorry, I wouldn’t expect Mac support any time soon. The Mac version of Mumble uses a completely different transport mechanism. Even if you could tap into the Cider win32 emulation layer, and create a memory mapped file, the GW2 code responsible may simply be disabled for safety when running on Mac. We are just not setup to test and support this scenario at this time.

It would be against our 3rd party app policy to scan the client in the manner you describe.

GW2's Mumble Link "context" data format

in API Development

Posted by: Felicja.6891

Felicja.6891

Thanks Cliff, this is very helpful. I appreciate your swift response.:)

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

Cliff, could please you provide us with some information about the character sets which are allowed (or better: forbidden, since the PCRE will match only ASCII chars) in character names so that we could e.g. run a regexp on them in order to store them in a DB or use them in scripts/apps/websites in a XSS/hack safe manner?

We know:

  • names are max 19 chars long
    * the most harmful chars (to HTML, also partly SQL) won’t ever appear, which are: & < > " ’ /
    * possibly and hopefully all other non-letter chars are forbidden too, such as backticks `, the hash sign # etc.

(dang, lists on this forum are so annoying >.<)

(edited by smiley.1438)

GW2's Mumble Link "context" data format

in API Development

Posted by: Cliff Spradlin.3512

Previous

Cliff Spradlin.3512

Lead Programmer

Next

Cliff, could please you provide us with some information about the character sets which are allowed (or better: forbidden, since the PCRE will match only ASCII chars) in character names so that we could e.g. run a regexp on them in order to store them in a DB or use them in scripts/apps/websites in a XSS/hack safe manner

I assume you’re asking about the JSON in the identity structure? JSON is UTF-8, and so any valid UTF-8 character is possible. You’ll need to do whatever you need to do to store and safely present any data, though I can’t really think of any possible attacks related to the mumble data.

It’s not clear to me how PCRE is related to any of this.

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

Sorry, i should have asked more clear: how would i validate a GW2 character name. See, when the mumble link data is used on web services like the one, Heimdall and me provide, it’s possible that also hackers try to break stuff e.g. with REST clients etc.. So it’s essential to validate any incoming data and/or sanitize it – thats what i already do. The only thing i’m unsure about is the character name. Check out my updated location receiver script to see what i’m doing:

https://github.com/codemasher/gw2api-tools/blob/master/examples/gw2location-receiver.php

(edited by smiley.1438)

GW2's Mumble Link "context" data format

in API Development

Posted by: poke.3712

poke.3712

I honestly can’t think of a good reason why you would want to validate a character name. Sanitizing data is not neccesary, if you pass it correctly. For databases use parameterized statements so that you don’t need to care about escaping ever; for HTML output escape control characters as usual.

Validating input is not trivial, especially when it’s about names. Also note that strlen is not aware of multibyte strings while GW2 supports those pretty well (we even can use umlauts!). And I would be careful making assumptions about global restrictions when you can just see those that apply for you (e.g. in GW1 I couldn’t enter Kanjis or Hanzis while others could enter those region-specific characters).

@poke // GW2W //

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

As you can see, i already use a prepared statement which is already pretty safe to store the data, but i’m umm… aware of unaware users ^.^ The most harmful characters are already blacklisted in the preg_match(), so this should be fine. Good point though on strlen(), i changed it to mb_strlen() instead.

GW2's Mumble Link "context" data format

in API Development

Posted by: Wothor.4781

Wothor.4781

So, I’m back from vacation and looking at the added info on idendity.
On first sight, it appears, world id and map id are duplicating the information from the context blob.
I’m just wondering if my original request, being the home world id (as opposed to guesting) got lost in the mail/translation.
Anti Spy won’t be possible without that, or a combination of smiley’s request for account name + a potentially upcoming API function to request that info for an account.

Given the voicecom espionage issues folks complain about I’d just thought hey, I could help out with that and additionally add quite adequate protection of the mumble data, as has beeen requested.
Personally I can live fine not having that, just pointing out the addition isn’t opening new options for that and am wondering if I should shelve that idea altogether with osx/linux users being excluded and the hassle of a plugin installation.

GW2's Mumble Link "context" data format

in API Development

Posted by: poke.3712

poke.3712

Anti Spy won’t be possible without that […]

Please note that the Mumble data can never be used as a safe authorization mechanism. It’s in a public memory location where I can write to from anywhere, so if I wanted to appear as someone else, I could easily do that, regardless of what information is included in the data.

So if you want to make sure that e.g. the people accessing a world’s data are actually from that world, you will have to use some other mechanism instead. For example a login with a white list or something; or if we get OAuth logins, we might be able to use the GW2 account directly (that also might be the time when we get further account information).

@poke // GW2W //

GW2's Mumble Link "context" data format

in API Development

Posted by: Wothor.4781

Wothor.4781

I was aware that anyone active in here could do that.
You could also have multiple accounts, then oauth bye bye.
It’s always just hurdles someone has to have the motivation and ability to break first.

GW2's Mumble Link "context" data format

in API Development

Posted by: Caldrick.6920

Caldrick.6920

You guys did a good job tracking down most of it.

Here’s the whole thing:


// uniquely identifies a map instance for mumble
struct MumbleContext {
    byte serverAddress[28]; // contains sockaddr_in or sockaddr_in6
    unsigned mapId;
    unsigned mapType;
    unsigned shardId;
    unsigned instance;
    unsigned buildId;
};

Cliff,
Would it be possible to add in which floor the character is currently at? Each map seems to have different distances between floors, and some floor distances don’t appear to be constant on some maps.

It looks like they are divided into 1000 inch increments from the couple of maps that I’ve looked at.

(edited by Caldrick.6920)

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

Now that the world/map display in friendlist etc. have been fixed, any chance that this will be also fixed for the mumble link data? I still receive “garbage” values when in WvW like 2147483655 for RoF borderland and 3221225491 for the EoTM instance i’ve been. Can’t you just return the actual home world id for players in WvW and overflows (e.g. 2005 in my case)?

GW2's Mumble Link "context" data format

in API Development

Posted by: Thuron.5648

Thuron.5648

Now that the world/map display in friendlist etc. have been fixed, any chance that this will be also fixed for the mumble link data? I still receive “garbage” values when in WvW like 2147483655 for RoF borderland and 3221225491 for the EoTM instance i’ve been. Can’t you just return the actual home world id for players in WvW and overflows (e.g. 2005 in my case)?

This!!! I tried to use those values for a WvW application a few days ago, but couldn’t because of the garbage values. Would be amazing if we can get the world-id inside WvW too.

http://www.gw2battlesupport.com/ – World’s first dynamic WvW Overlay

GW2's Mumble Link "context" data format

in API Development

Posted by: Reincarnated.1754

Reincarnated.1754

I done a little program to help me remember some of the events I have done each day.
https://www.dropbox.com/s/48vrruqyilizt4y/Boss%20Timer.rar?dl=0

There is a map section that uses the API. Helps me in WvW

I have found *identity * to be:
{"name": “Palingenesis”,“profession”: 6,“map_id”: 18,“world_id”: 268435465,“team_color_id”: 0,“commander”: false}

name — that straight forward
profession — I think 1=Guardian, 2=Warrior, 3=Engineer, 4=Ranger, 5=Thief, 6=Elementalist, 7=Mesmer, 8=Necromancer.
map_id — this works with https://api.guildwars2.com/v1/maps.json?map_id=18
world_id — this I don’t know, any one able to help with the number given here? (268435465)
team_color_id — I think 0=Red, 1=Green, 2=Blue.
commandertrue if you are commanding.

So do anyone know what the world_id number represents?

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

The world_id is (or should be) your home world id which can (could) be assigned to https://api.guildwars2.com/v1/world_names.json . However since the megaservers were introduced it’s basically messed up and will only return usable values in case you’re in WvW.

The team_color_id value isn’t 0/1/2 but 0 for neutral and the actual color_id from https://api.guildwars2.com/v1/colors.json for red/blue/green.

(edited by smiley.1438)

GW2's Mumble Link "context" data format

in API Development

Posted by: StevenL.3761

StevenL.3761

268,435,465 decimal is 10 00 00 09 hexadecimal. Does that number ever change? What happens when you guest on another world?

GW2's Mumble Link "context" data format

in API Development

Posted by: draos.9574

draos.9574

The world_id is broken with the megaserver system. Only working in wvw but there which some problems.

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

@Devs:

Will there be an update to the context data to reflect the changes of the commander icon colors? (like commander: 0 = off, 1 = blue, 2= red, …)
Also: can you please fix and/or explain the “garbage” values which are currently returned as world_id?

GW2's Mumble Link "context" data format

in API Development

Posted by: meh.6784

meh.6784

The world_id seems to be broken in WvW as well.

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

The world_id seems to be broken in WvW as well.

Right. I forgot that the world_id was always broken in WvW, so this value is basically useless now in all cases.

GW2's Mumble Link "context" data format

in API Development

Posted by: meh.6784

meh.6784

The world_id seems to be broken in WvW as well.

Right. I forgot that the world_id was always broken in WvW, so this value is basically useless now in all cases.

Aww that sucks, hope they fix it so I can find the world automatically from my Overwolf app.

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

The world_id seems to be broken in WvW as well.

Right. I forgot that the world_id was always broken in WvW, so this value is basically useless now in all cases.

Aww that sucks, hope they fix it so I can find the world automatically from my Overwolf app.

soon™

GW2's Mumble Link "context" data format

in API Development

Posted by: StevenL.3761

StevenL.3761

Has anybody gathered any new information over the last couple of weeks? I already know that the value of “world_id” is now set to the same value as “shardId” of struct “MumbleContext”. It appears that this value is to be used as a bit mask, not an actual ID. I can confirm that the “shardId” and “worldId” change as you move between different parts of the game, but I haven’t cracked the pattern yet.

GW2's Mumble Link "context" data format

in API Development

Posted by: StevenL.3761

StevenL.3761

Some shard_id values that I recorded so far (logged in from the Ring of Fire server):

10000000000000000000000000000111, Eternal Battlegrounds
11000000000000000000000000000101, Edge of the Mists
10000000000000000000000000000111, Borderlands
00010000000000000000000000000100, Dry Top
00010000000000000000000000000001, Queensdale
00010000000000000000000000000001, Gendarran Fields
00010000000000000000000000001011, Harathi Hinterlands
00010000000000000000000000000001, Lornar’s Pass
00010000000000000000000000001010, Divinity’s Reach
00010000000000000000000000001001, Lion’s Arch
00010000000000000000000000000001, The Grove
00010000000000000000000000000010, Rata Sum
00010000000000000000000000000001, Black Citadel
00010000000000000000000000000001, Hoelbrak
00010000000000000000000000000001, Southsun Cove
00010000000000000000000000011111, Sparkfly Fen
00010000000000000000000000000011, Straits of Devastation
00010000000000000000000000000001, Caledon Forest
00000000000000000000011111010101, Twilight Arbor

There is definitely a pattern here. I’m just not seeing it. Maybe if we can complete the data for all maps.

GW2's Mumble Link "context" data format

in API Development

Posted by: smiley.1438

smiley.1438

00000000000000000000011111010101, Twilight Arbor

11111010101 → 2005 decimal which is RoF’s world_id (coincidence?)

GW2's Mumble Link "context" data format

in API Development

Posted by: StevenL.3761

StevenL.3761

That makes sense. Dungeons are instanced, so they wouldn’t put them on a megaserver shard.

I still don’t know about the other maps though.

GW2's Mumble Link "context" data format

in API Development

Posted by: Dr Ishmael.9685

Dr Ishmael.9685

The leftmost bits look somewhat like a continent ID.

0000 → dungeons (and maybe other instances as well)
0001 → Tyria
1000 → WvW
1100 → EotM

GW2's Mumble Link "context" data format

in API Development

Posted by: StevenL.3761

StevenL.3761

Could the first 4 bits actually be a map type instead of a continent ID? What makes EotM different from other WvW maps?

0000 → Instance
0001 → PvE
1000 → WvW
1100 → WvW (EotM)
???? → PvP

The MumbleContext struct has a mapType property that may come in use. I don’t have any data on it though.

GW2's Mumble Link "context" data format

in API Development

Posted by: StevenL.3761

StevenL.3761

Some map types that I was able to gather (MumbleContext mapType):

2 = PvP maps
4 = Instanced map (dungeons, story, home instance, activity)
5 = PvE (all explorable zones except Dry Top)
9 = Eternal Battlegrounds
10 = Blue Borderlands
11 = Green Borderlands
12 = Red Borderlands
15 = EotM
16 = Dry Top

I wouldn’t be surprised if there are more map types, one for every number not already on my list (1, 3, 6, 7, 8 ,13, 14).

GW2's Mumble Link "context" data format

in API Development

Posted by: Reincarnated.1754

Reincarnated.1754

identity has been updated.
They added race and fov

race:
0 = Asura
1 = Charr
2 = Human
3 = Norn
4 = Sylvari

I asume fov is Field of View, can some one confirm?

GW2's Mumble Link "context" data format

in API Development

Posted by: Lawton Campbell

Previous

Lawton Campbell

Web Programmer

Next

Yeah, fov is the current field of view.

GW2's Mumble Link "context" data format

in API Development

Posted by: aRestless.6213

aRestless.6213

Just to be save: Are we talking about the horizontal, vertical, or diagonal field of view?

Never lose track of your friends again, with Who’s Talking for Overwolf

GW2's Mumble Link "context" data format

in API Development

Posted by: Lawton Campbell

Previous

Lawton Campbell

Web Programmer

Should be the vertical FOV.

GW2's Mumble Link "context" data format

in API Development

Posted by: MentalFS.2589

MentalFS.2589

race:
0 = Asura
1 = Charr
2 = Human
3 = Norn
4 = Sylvari

No new race alphabetically placed before Sylvari confirmed. Conveniently, there are races whose name starts with letters like T

Let the bards sing and the skaalds proclaim her glorious name forever[…]: Apatia!