Wish: Better searching

Wish: Better searching

in API Development

Posted by: Killerassel.2197

Killerassel.2197

There are two things I have noticed lately on using the API. I’d like to be able to find an item by its name and the recipes by discipline and possibly min_rating. An example use case: which recipes for scribes use T1 blood?

Of course it would be even better the more general the search options are. Ideally it would be possible to find by every aspect that is included in the result, such as items by type, flags, details, rarity etc. For example: find all level 80 heavy armor boots that give power (and any other stats).

This would rather be a convenience (and performance) feature. To get my list of scribing recipes I just pulled all the recipes and filtered them by discipline. The same approach can be used to find an item by name. And of course I could cache it in a local database. But I think it would be nice if one could just ask the API for it directly.

Oh, and item by name should at least optionally be partial name, just like on the TP.

(edited by Killerassel.2197)

Wish: Better searching

in API Development

Posted by: DarkMagister.7429

DarkMagister.7429

I think let not try to sit on anet’s head here )) They are providing us with some nice info to use, but if all the sites or programs start requesting anet’s server for all these things – it would not be so good.

“which recipes for scribes use T1 blood?” – way I remember there is an API to return the list of recipes using the item, then you can go through that short list to check which of them is for scribe.

BUT I doubt that API provides recursive check, nor will any other API of that kind. I mean “A is used for crafting B, B is used for crafting C. You won’t get C in the response list for A”. So it is something you will still need to do on your side.

Wish: Better searching

in API Development

Posted by: Lawton Campbell

Lawton Campbell

Web Programmer

Next

This is something we’ve looked into but are unlikely to support, unfortunately.

Wish: Better searching

in API Development

Posted by: Killerassel.2197

Killerassel.2197

DarkMagister, you’ve read too much into my post. The API already supports listing recipes by (immediate) ingredient or by result. It takes the item ID as parameter, though. So, what I need is a way to translate the name of an item into its ID, i.e. a search item by name function.

I’d consider recursion through recipes my application’s job anyway.

My problem with with the API as it is now is, that I cannot search an item or a recipe by specific properties directly. For recipes specifically, I can ask by ingredient or result, but I cannot filter by discipline or min-level.

I.e. I CAN ask “show all recipes using item 12345”, but I cannot ask “show all recipes for scribes (using item 12345)”.

The workaround that I can use is pulling ALL items/recipes and filter locally.

Unfortunately this workaround will have to do for the forseeable future. Thanks for the feedback anyway and keep up the nice work.

Wish: Better searching

in API Development

Posted by: DarkMagister.7429

DarkMagister.7429

Maybe I’ve read too much, but I meant what it:
ANet is interested in moving as much search job to our side, instead of loading its servers. Thus it is reasonable for them to leave such complex searches to our side.
(this may not be a problem on their side, but I see some requests dropped in the prime-time already)

And if you already consider recursion to be done on your side anyway, why do you need any search at the server-side? To first get recipe-list from the server, then use your recursive function (returning recipes, using this item) on the results from this list – it is counter-productive.

P.S. Trying to use recursive search on MF recipes will need additional fail-safes added.

Wish: Better searching

in API Development

Posted by: Killerassel.2197

Killerassel.2197

At the recursion stage I’m already working with IDs. At that time I won’t need (other) search conditions anymore.

Consider the following two questions and how you’d answer them from the API:

  • What can I craft from mithril ore?
  • What recipes can my Scribe (level xx) craft?

There is already /recipes/search?input=…, which allows to query “What can I craft from the item with ID 1234?” So to find out what I can craft from mithril ore I need to find its ID first. And this is not directly possible via the API, other than loading ALL items and check each item’s name.

Similar for the second question. I can get a list of all recipes and each recipe states its discipline(s). But I cannot query, say, /recipes/search?discipline=Scribe.

Once I have recipes I need to find what the item IDs actually represent and possibly how to craft them. This is the recursion step, but this can be done without search but directly querying items by their ID and recipes by /recipes/search?output=…, which exists.

To put it in SQL terms, I’d like to have more options/indexes for WHERE conditions, but my post was not at all about JOIN (i.e. recursion).

And about the server load: well, it’s a trade off. My suggestion would put a bit more load for processing, but take away a lot of load on bandwidth. Unless, of course, high bandwidth sites start to abandon their local caches and start hammering the server.

Wish: Better searching

in API Development

Posted by: Lawton Campbell

Previous

Lawton Campbell

Web Programmer

To put it in SQL terms, I’d like to have more options/indexes for WHERE conditions, but my post was not at all about JOIN (i.e. recursion).

The API doesn’t talk to any relational databases; the recipe search endpoints are using hand-rolled indexes. This doesn’t really scale in a sane way; it would be ideal to toss all of the data into a more production-ready indexing solution (read: hand-normalize it into SQL, or toss is into an indexed document store ala Lucene) but we don’t really have the resources to set up and maintain additional third-party software solutions.

Wish: Better searching

in API Development

Posted by: Arridath.1087

Arridath.1087

The API doesn’t talk to any relational databases; the recipe search endpoints are using hand-rolled indexes. This doesn’t really scale in a sane way; it would be ideal to toss all of the data into a more production-ready indexing solution (read: hand-normalize it into SQL, or toss is into an indexed document store ala Lucene) but we don’t really have the resources to set up and maintain additional third-party software solutions.

Oh my god I am so sorry this is the case.

With regards to OP’s request, the GW2Spidy API might be of use. They poll the official API and keep all of their data in a separate database. Because of this, they support searching for an item by name.

Wish: Better searching

in API Development

Posted by: Killerassel.2197

Killerassel.2197

The API doesn’t talk to any relational databases;

I had almost expected that. But I thought putting it in familiar SQL terms anyway helped me to clarify what I’m asking for and clear up the confusion DarkMagister pointed at.

This doesn’t really scale in a sane way; it would be ideal to toss all of the data into a more production-ready indexing solution (read: hand-normalize it into SQL, or toss is into an indexed document store ala Lucene) but we don’t really have the resources to set up and maintain additional third-party software solutions.

Ouch. Ok, but since there is an index to find recipes by ingredient and output, at least there was some hope this would be doable.

With regards to OP’s request, the GW2Spidy API might be of use.

Thank you for the suggestion. Yes, this would be another good option.

Wish: Better searching

in API Development

Posted by: DarkMagister.7429

DarkMagister.7429

Seems like things are more evident now, but since it was asked.

Consider the following two questions and how you’d answer them from the API:

  • What can I craft from mithril ore?
  • What recipes can my Scribe (level xx) craft?

I develop my tool as an application, not a web-project, so for me running though some lists or dictionaries looks like a faster way to get things done, then requesting server for each action.

Question 1: It can be split into two different actions, first being an addition to the second:
1. Connect user input “Mithril Ore” with the item ID (if it is a user input and we don’t know the ID). I’ve added it not so long ago, so: I have two lists: one if integer – IDs, second of string (item names). The search function creates the list of indexes of items in the 2nd list, containing “Mithril Ore”, then converts it into the list of item IDs from the 1st list (item #10 corresponds to the item #10), so the list of item IDs is returned to the table, from which Mithril Ore can be selected – we get our ID.
2. Code:

Public Function GetRecipesUsingItem(i_ItemID As Integer) As List(Of cls_HolderAPI2RecipeDetails)
        Dim lst_return As New List(Of cls_HolderAPI2RecipeDetails)
        For Each item In dic_RecipesDetails
            For Each item2 In item.Value.ingredients
                If item2.item_id = i_ItemID Then
                    Dim lst_LvlUPRecipes As New List(Of cls_HolderAPI2RecipeDetails)
                    lst_LvlUPRecipes = GetRecipesUsingItem(item.Value.output_item_id)
                    If lst_LvlUPRecipes IsNot Nothing Then
                        lst_return.AddRange(lst_LvlUPRecipes)
                    End If
                        lst_return.Add(item.Value)
                End If
            Next
        Next
        If lst_return.Count = 0 Then
            Return Nothing
        Else
            Return lst_return
        End If
End Function

This recursive function returns the list of all recipes, in which the given item and its “children” are used as ingredients. Then another function converts it into the list of results item IDs, which is then converted into names at the stage of filling the result table. It can be converted directly, but the program works with IDs, not names, I only use names when it is something the user is to see.

Conversion to name is simple, item details are stored in the dictionary of item details classes, with item IDs being keys. So, for example
[pre]msgbox (GetItemForID(i_ItemID).name)

Public Function GetItemForID(i_ID As Integer) As cls_HolderAPI2ItemDetails
If dic_ItemsDetails.ContainsKey(i_ID) = True Then
Return dic_ItemsDetails.Item(i_ID)
End If
Dim cls_ZeroResult As New cls_HolderAPI2ItemDetails
cls_ZeroResult.name = “N/A”
Return cls_ZeroResult
End Function[/pre]

“What recipes can my Scribe (level xx) craft?”
I have a boolean function b_IsNeededDiscipline, that checks: is the given recipe corresponds to the given recipe search settings (discipline and level), like Scribe (level xx) . It is used on several occasions, in your task it will be cycling through all the recipes, creating the list of recipes IDs, that correspond to the given setting.

I’m sorry for much text, and possibly sorry for stupid code – I’m not a pro.

P.S. I understand that using a local database would be the right way to do it, but had my reasons to do it this way.

(edited by DarkMagister.7429)