Places query syntax

Discussion about official Mozilla Firefox builds
Locked
User avatar
Max Karl Ernst
Posts: 107
Joined: February 5th, 2008, 3:43 am

Places query syntax

Post by Max Karl Ernst »

This is a discussion on serialized 'places:' query strings.
Firefox 3 switched to "places" sqlite database as a main bookmarks and history container. That brings great advantage to FF users, but unfortunately lack of time left most of that database power under the hood with FF3. Still, you can get a glimpse of SQL query power by using serialized 'places:' query strings, simple strings that can be added to your bookmarks that will perform queries over database. You already have some of them, like "most visited", "recently bookmarked" and such.

Here is the basic knowledge about query system that we collected so far and few examples.
This list comes from a joint effort of mZ forum posters and is not necessarily correct or in way comprehensive :D


QUERY STRING EXAMPLES

Default smart bookmarks (for those that incidentally deleted them):
Most Visited

Code: Select all

place:queryType=0&sort=8&maxResults=10

Recently Bookmarked

Code: Select all

place:folder=BOOKMARKS_MENU&folder=UNFILED_BOOKMARKS&folder=TOOLBAR&queryType=1&sort=12&excludeItemIfParentHasAnnotation=livemark%2FfeedURI&maxResults=10&excludeQueries=1

Recent Tags - sorted by last modified date

Code: Select all

place:type=6&sort=14&maxResults=10

Unsorted Bookmarks

Code: Select all

place:folder=UNFILED_BOOKMARKS&excludeItems=1&expandQueries=0


More examples (chronologically):
Query for all history from last day that contains "firefox" and is from mozillazine.org domain
Query for all bookmarks with "top" tag (broken)
Query for term "linux"(broken)
Query for most visited bookmarked items
Query for items with recently used tags
Default history and default bookmarks menu
Add Unfiled Bookmarks to Bookmarks Menu
Retreive bookmarks with specific tag (doesn't use terms, but folder id)


BUILD YOUR OWN QUERY

1. Query search parameters:
(All parameters are ANDed together)

beginTime (Unsigned Long)
Time range for results. Lower boundary.
Values:
Any. Time format is in microseconds, so multiply with 1000000 * 60 * 60 for hours and for days multiply that with 24.

beginTimeRef (Unsigned Long)
Indicates how to interpret the corresponding time value in beginTime.
Values:
0 (default) - The time is relative to Jan 1 1970 GMT
1 - The time is relative to this morning at midnight. Normally used for queries relative to today.
2 - The time is relative to right now.

endTime (Unsigned Long)
Time range for results. Upper boundary.
Values: Same as beginTime.

endTimeRef (Unsigned Long)
Indicates how to interpret the corresponding time value in endTime.
Values: Same as beginTimeRef

terms (String)
Query this term
Values: Any

minVisits (Long)
Filter results based on number of visits. This is lower value
Values:
-1 (default) - Show all
Any - Show items with visit count larger than this number.

maxVisits (Long)
Filter results based on number of visits. This is upper value
Values:
-1 (default) - Show all
Any - Show items with visit count smaller than this number.

onlyBookmarked (Boolean)
Return only bookmarked items
Values:
true - Return only bookmarked items
false - Return all

domain (String)
Query for this host or domain name (controlled by domainIsHost).
Values: Any

domainIsHost (Boolean)
Controls the meaning of 'domain'
Values:
true - exact match
false - hierarchical (Returns anything whose host name ends in 'domain')

uri (URI)
URI to match
Values: Any

uriIsPrefix (Boolean)
Controls the interpretation of 'uri' (above).
Values:
false (default) - the URI will request an exact match of the specified URI.
true - Any history entry beginning in 'uri' will match (for example "http://bar.com/foo" will match "http://bar.com/foo" as well as "http://bar.com/foo/baz.gif")

annotation (String)
Annotation to match
Values: Any

!annotation (Boolean)
Test for existance or non-existance of a given annotation. "Annotation is not"
Values:
true - Remove everything that doesn't have annotation.
false - Normal results

folder (String)
Folder that is queried
Values:
PLACES_ROOT - Places Root Folder
BOOKMARKS_MENU - Bookmarks Menu
TOOLBAR - Bookmarks Toolbar
TAGS - Tags
UNFILED_BOOKMARKS - Unfiled Bookmarks

2. Configuration of the search result

OR
Logical operator

sort (Unsigned Short)
Sort results
Values:
0 - Natural bookmark order
1 - Sort by title, A-Z
2 - Sort by title, Z-A
3 - Sort by visit date, most recent last
4 - Sort by visit date, most recent first
5 - Sort by uri, A-Z
6 - Sort by uri, Z-A
7 - Sort by visit count, ascending
8 - Sort by visit count, descending
9 - Sort by keyword, A-Z
10 - Sort by keyword, Z-A
11 - Sort by date added, most recent last
12 - Sort by date added, most recent first
13 - Sort by last modified date, most recent last
14 - Sort by last modified date, most recent first
17 - Sort by tags, ascending
18 - Sort by tags, descending
19 - Sort by annotation, ascending
20 - Sort by annotation, descending

sortingAnnotation (String)
The annotation to use in sort by annotation modes

type (Unsigned Short)
Type of results that are returned
Values:
0 - Results as URI ("URI" results, one for each URI visited in the range).
1 - Results as visit ("visit" results, with one for each time a page was visited - this will often give you multiple results for one URI).
2 - Results as full visits (like "visit", but returns all attributes for each result)
3 - Results as date query (returns results for given date range)
4 - Results as site query (returns last visit for each url in the given host)
5 - Results as date+site query (returns list of hosts visited in the given period)
6 - Results as tag query (returns list of bookmarks with the given tag)
7 - Results as tag container (returns bookmarks with given tag - for same uri uses last modified) - folder=tag_folder_id must be present in the query

excludeItems (Boolean)
Exclude all URIs and separators from the bookmark query results (ignored for queries over history.)
Values:
true - Return only list of bookmark folders and queries (such as the left pane of the places page)
false (default) - Return full list

excludeQueries (Boolean)
Excludes queries from the query results (simple folder queries like bookmark folder symlinks will still be included.)
Values:
true - Return list without queries
false (default) - Return full list

excludeReadOnlyFolders (Boolean)
Excludes read-only folders from the query results (this only affects cases where the actual folder result node would appear in its parent folder and filters it out)
Values:
true - Return list without read-only folders
false (default) - Return full list

excludeItemIfParentHasAnnotation (String)
Exclude item whose parent has this annotation from the bookmark query results (ignored for queries over history.)An example is to exclude livemark items (parent folders have the "livemark/feedURI" annotation)
Values:
Any

expandQueries (Boolean)
Allows items with "place:" URIs to appear as containers,with the container's contents filled in from the stored query (doesn't do anything if excludeQueries is set).
true - "place:" URIs appear as containers
false (default) - "place:" URIs appear as normal items

includeHidden (Boolean)
Returns normally hidden items in history query (content of iframes and all images on web pages). Does nothing for bookmark queries
Values:
true - Return items including hidden ones
false (default) - Return normal results

showSessions (Boolean)
Separate/group history items based on session information. Only matters when sorting by date.
Values:
true - Use session information when returning results
false (default) - Normal results

maxResults (Unsigned Long)
Set max number of results to return. This doesn't work in conjunction with sorting by title!
Values:
0 (default) - Get all results
Any: The query is executed, the results are sorted, and then the top 'maxResults' results are taken and returned.

resolveNullBookmarkTitles (Boolean)
If a bookmark title is NULL (note, not empty), attempt to use the history title.
Values:
true - Attempt
false (default) - Don't attempt

queryType (Unsigned Short)
The type of search to use when querying the DB; This attribute is only honored by query nodes. It is silently ignored for simple folder queries.
Values:
0 - History
1 - Bookmarks
2 - Unified

originalTitle (String)
Retrieves original page title.
Values:none?


MORE INFORMATION

Detailed explanations on some of query strings.
Places Documentation
MDC wiki - Places query URIs
MDC wiki - Querying Places
Source: nsINavHistoryService.idl
Source: nsNavHistory.cpp
Source: utils.js
Source: nsBrowserGlue.js


IMPORTANT RELATED BUGS (to be updated)

Bug 399799 – Provide better options to query for tags - VERIFIED FIXED
Bug 411591 – expose frecency as a sort order for place queries - NEW
Bug 420282 – After bug 385245, need RESULTS_AS_FOLDER_QUERY result type - UNCONFIRMED
Bug 421118 – break inheritance of the sorting mode - ASSIGNED
Bug 421953 – Implement RESULTS_AS_DATE_HOST_SITE_QUERY - NEW
Bug 426870 – Bookmarking place: URIs can lead to circular menus - NEW
Bug 430580 – User defined Saved Search should be editable - NEW
Bug 430702 – Bookmarked place: URIs are not consistently presented - NEW


TOOLS

Patch For Library - by White Alice0775.
With FF3 final, the query editing functionality in Library has been disabled and pushed back for future versions. This extension brings that functionality back, fixes lots of bugs and enables you to see tag folder id's.
SearchPlaces - by Andy Halford.
You can now use this tool to create and edit complex queries.
Last edited by Max Karl Ernst on June 2nd, 2009, 2:31 am, edited 36 times in total.
Oropher
Posts: 346
Joined: September 26th, 2007, 7:12 pm

Post by Oropher »

I tried working out the query system a while ago, but at the time I kept hitting unimplemented features. Not sure if I still have the queries I worked out, either.
Dietrich has a Places ERD up somewhere, but I don't have a link right now. I'll see what I can dig up ;)
Edit: found it on <a href="http://wiki.mozilla.org/Places:Design_Overview#Models">wiki.mozilla.org</a>.
User avatar
Max Karl Ernst
Posts: 107
Joined: February 5th, 2008, 3:43 am

Post by Max Karl Ernst »

Great :)

All I can see right now is

folder=2 - Bookmarks Menu
folder=3 - Bookmarks Toolbar
folder=4 - Tags
folder=5 - Unfiled Bookmarks

(One quick usage that springs to mind, if you add bookmark place:folder=5 there's your quick access to unfiled bookmarks)

And obviously sort=12 is sort by time and sort=8 is sort by visit count. All other is darkness :)

[edit] Thanks for link I'll see if I can get some clue :)
Oropher
Posts: 346
Joined: September 26th, 2007, 7:12 pm

Post by Oropher »

Quick summary of what I can remember:

terms= - Search term

folder=0 - ?
folder=1 - ?
folder=2 - Bookmarks Menu
folder=3 - Bookmarks Toolbar
folder=4 - Tags
folder=5 - Unfiled Bookmarks

queryType=0 - History
queryType=1 - Bookmarks

group=0 - group by days since last visit
group=1 - group by subdomain & domain
group=2 - group by domain
group=3 - group by tag

sort=0 - ?
...
sort=8 - sort by visit count, descending
...
sort=12 - time, most recent first

Edit: Oops, see you got there first :)
Last edited by Oropher on February 18th, 2008, 6:35 am, edited 1 time in total.
User avatar
Max Karl Ernst
Posts: 107
Joined: February 5th, 2008, 3:43 am

Post by Max Karl Ernst »

Actually I've missed "terms" can you give me a hint how that one works because that's exactly one I've been searching for the most :)

Also, thanks for resolving "queryType" and "group" :)
Oropher
Posts: 346
Joined: September 26th, 2007, 7:12 pm

Post by Oropher »

OK, 'terms' is the search term. It doesn't seem to work without some combination of '&excludeQueries=1&expandQueries=0' on the end (at least, it always returns empty). I don't know what those extra terms do, 'cause last time I was looking into this MDC said they weren't implemented yet.

Edit: be wary of trying 'group=3' with 'terms' - it just asserted like mad for me and I had to kill it.

Edit2: 'group=0' sorts into folders based on last visit date - '2 days ago', '4 days ago', 'Older than 6 days'; 'group=1' sorts by domain and subdomain; 'group=2' sorts by domain. However, I can't access these subfolders in the Library, and the Bookmarks Menu isn't showing them at all...
User avatar
Max Karl Ernst
Posts: 107
Joined: February 5th, 2008, 3:43 am

Post by Max Karl Ernst »

Great :)

I couldn't get "terms" to work, until I restarted firefox :)

So now I can add this bookmark to my Bookmark Toolbar

Code: Select all

place:folder=4&queryType=1&group=3&terms=top&sort=1&excludeQueries=1&expandQueries=0


and any bookmark I tag "top" would be a click away. Neat :)

[edit] sort=1 sorts by Title. Editing to include that finding :)
Last edited by Max Karl Ernst on February 18th, 2008, 7:44 am, edited 1 time in total.
Oropher
Posts: 346
Joined: September 26th, 2007, 7:12 pm

Post by Oropher »

Max Karl Ernst wrote:Great :)

I couldn't get "terms" to work, until I restarted firefox :)

So now I can add this bookmark to my Bookmark Toolbar

Code: Select all

 place:folder=4&queryType=1&group=3&terms=top&excludeQueries=1&expandQueries=0


and any bookmark I tag "top" would be a click away. Neat :)

Does that give you the bookmark title or the page title?
User avatar
Max Karl Ernst
Posts: 107
Joined: February 5th, 2008, 3:43 am

Post by Max Karl Ernst »

Oropher wrote:Does that give you the bookmark title or the page title?


It returns whatever all other tags return, I'm not sure what is bookmark and what is page title :)
Whatever it is, it is still under the same bug that doesn't return proper titles.
Cye3s
Posts: 20
Joined: December 28th, 2007, 9:53 pm

Post by Cye3s »

Great!

Code: Select all

place:beginTime=-86400000000&endTime=0&domain=mozillazine.org&terms=firefox&queryType=0


It return all historys that page title with firefox and domain is mozillazine.org today(24 hours).
0 is now,86400000000=24 * 60 * 60 * 1000000
Cye3s
Posts: 20
Joined: December 28th, 2007, 9:53 pm

Post by Cye3s »

Oropher
Posts: 346
Joined: September 26th, 2007, 7:12 pm

Post by Oropher »

Here's an updated list of the 'sort' options. It's not complete yet, I'm still getting different results for higher numbers, just can't figure what they're sorting by yet.

sort=
0 - unsorted?
1 - sort by name (bookmark or page or url) A-Z
2 - sort by name Z-A
3 - visit date, most recent last
4 - visit date, most recent first
5 - uri, A-Z
6 - uri, Z-A
7 - visit count, ascending
8 - visit count, descending
9 - keyword, A-Z
10 - Keyword, Z-A
11 - date added, most recent last
12 - date added, most recent first
13 - last modified, most recent last
14 - last modified, most recent first
User avatar
Max Karl Ernst
Posts: 107
Joined: February 5th, 2008, 3:43 am

Post by Max Karl Ernst »

You guys are superb, I'll edit first post to include all our findings :)
Oropher
Posts: 346
Joined: September 26th, 2007, 7:12 pm

Post by Oropher »

A bit more:
15 - apparently =1, but NOT sorting on Name field
16 - apparently =2, but NOT sorting on Name field
17 - sort by tags, A-Z
18 - sort by tags, Z-A

About 15 & 16, the results seem to be sorted alphabetically like 1 & 2, but the Library doesn't indicate that it's sorting on the Name field so it's probably not simply a duplicate. At this point I've no idea what it's actually sorting by. I don't have an SQLite reader on Linux at the moment, so that'll just have to wait.
bomfog
Posts: 455
Joined: November 7th, 2002, 3:22 pm
Location: the palouse, SE. WA, USA

Post by bomfog »

Perhaps see also nsINavHistoryService.idl, especially around line 867 and below.
--
Mike
Locked