Cache Busting Wikidata SparQL Queries

26th January 2018

UPDATE: By setting a cache-control: no-cache header you can disable this query caching.

The problem

Whenever you write a Wikidata query form which you do not expect the result to be the same each time, you will run into the issue of caching. Lets take the following example, returning a random cat:

SELECT ?item ?itemLabel (MD5(CONCAT(STR(?item), STR(RAND()))) as ?random)
WHERE 
{
  ?item wdt:P31 wd:Q146.
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE], en". }
} ORDER BY ?random 
LIMIT 1

Run in the Wikidata Query Editor

If you run this once you will get a random cat if you run this twice you will get the same random cat.

This is because Wikidata has saved the result, to serve you faster the second time.

The Solution

A solution some people uses is to “just add a space somewhere”, this is possible because Wikidata cache queries based on the SparQL string, not the actual parsed and preformed query.

When used in implementations keeping track of spaces is not an option so I decided to use random comments that could be generated from hashing functions etc:

#01e8c03a6bdfe392431d8189130fcfc0
SELECT ?item ?itemLabel (MD5(CONCAT(STR(?item), STR(RAND()))) as ?random)
WHERE 
{
  ?item wdt:P31 wd:Q146.
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE], en". }
} ORDER BY ?random 
LIMIT 1

Changing the comment string and rerunning the query will result in an new random cat.

When actually used in real world implementations where you might minify your queries I suggest appending the comment just before preforming it.

Final Notes

Build Something with K-Samsök and Python

25th July 2017

Last week I took a day to do a bit of open source maintenance, fixing bugs, writing documentation, tests etc and definitive no new features.

One of the things I did was to write documentation for my K-Samsök Python library KSamsok-PY. It’s a port of KSamsok-PHP which I have written a lot about. Nowadays I have a local setup of kulturarvsdata.se (the repository behind K-Samsök), so the libraries is mainly used in production and to maintain my local instance. KSasmok-PY is actually powering Kyrksök.

pip install ksamsok and head over to the documentation.

Kyrksok.se gets a VR Viewer

18th June 2017

The Kyrksok logo

Kyrksok.se is a directory for churches in Sweden that was originally created last autumn over a weekend. Last week I took the opportunity to add a VR Viewer for 360 degrees photos to the site.

It’s primarily built on top of three.js and WebVR-UI and you can try it out here.

It’s using the WebVR API behind the scenes if you have a headset or Cardboard if not there is a fall-back for mouse and mobile devices.

One thing I would like to do with WebVR in the future is to play around with Mozillas A-Frame library together with OpenStreetMap 3D data, but for now I’m awaiting someone to put a VR-headset in my mailbox :-)

Extending Native Context Menus with HTML 5.1 <menu>

15th June 2017

One thing that annoys me when using the web is when some developer decided to do preventDefault() on contextmenu to replace the native context menu with some shitty app specific one, blocking my access to thing such as spellcheck, plugins, and search. It’s something I believe should be avoided whenever possible. Therefor I’m a fan of the possibility to extend the native one.

It should be noted that this is only available in Firefox for the moment.

My native context menu is full of useful features:

screenshot

I added items to the context menu to my “bookshelf” now when right clicking a book you have the options to search after the book title at Amazon or Bokus.se as well as go back to byabbe.se.

Custom parts of the native context menu is defined using the <menu> element(<menu> is standardized to define menus for buttons and toolbar too), to link a <menu> to its container there is an contextmenu attribute, I added this attribute to the element containing all books as children.

<menu type="context" id="book-menu">
  <menuitem label="Go to byabbe.se" id="byabbe-se"></menuitem>
  <menu label="Search">
    <menuitem label="Bokus" id="bokus-search"></menuitem>
    <menuitem label="Amazon" id="amazon-search"></menuitem>
  </menu>
</menu>

Adding actual functionality to the items is done with the regular click event. But in this case the action deepened upon which element(book) that the context menu had been triggered on. This was some thing I had too keep track of by another listener.

let currentContext = '';
document.querySelectorAll('.list')[0].addEventListener('contextmenu', e => {
  let target = e.target;
  let bookContainer;
  switch (e.target.tagName) {
    case 'SPAN':
      currentContext = target.parentElement.parentElement.children[0].innerText;
      break;
    case 'P':
      currentContext = target.parentElement.children[0].innerText;
      break;
    case 'B':
      currentContext = target.parentElement.children[0].innerText;
      break;
    case 'LI':
      currentContext = target.children[0].innerText;
      break;
    default:
      currentContext = target.innerText;
  }
});

document.querySelector('#amazon-search').addEventListener('click', e => {
  document.location = 'https://www.amazon.com/s?field-keywords=' + currentContext;
});

document.querySelector('#bokus-search').addEventListener('click', e => {
  document.location = 'http://www.bokus.com/cgi-bin/product_search.cgi?search_word=' + currentContext;
});

You can try it out in Firefox over at byabbe.se/books/.

screenshot

I believe there at least one major flaw with the <menu> element because you add items to the root of the menu, and there is no limit to the amount of items you can define. This results in tons of possible abuses such as using custom menu items to push the browser specific ones of screen(I tried, it works). It would be a lot better for the end user if all custom items would be one level below the root.

Older