Tester: use more efficient RecordSpaces

[evert]

To avoid conflicts between multiple workers on the same records from a Record space, in revision 5370, the record spaces have been made worker-local.
This is however not yet ideal. The number of records in a recordspace is now multiplied by the number of workers, which is something that is not controlled in the configuration file.

Possible solution:
Change back to using one record space for all workers.
Change the behaviour of the update action: It first removes a record it is going to work on (avoiding that other workers will try to update the same record) and afterwards put back the record to the destination recordspace. It is up to the user to define the destination records space the same as the source record space or as a new record space.

Issue:
When doing more updates or deletes than creates, it is possible that the record space would get empty. We could either throw an exception then, or skip the action when no record is to be found (my preference). The metrics will only measure actual executed actions. One should not make the conceptual mistake of thinking that when x updates or deletes are defined in the scenario, that there will be x updates or deletes effectively executed.
While 'skipping the action' we should avoid that it is a busy wait, burning CPU cycles.

The RecordSpace? is currently implemented with a ListOrderedSet? to be able to pick a random record and replace it with a new record. This is however not efficient: both a List and a Set are being updated, and the elements of the list need to be moved each time an element is removed somewhere in the middle.
Possible solution: just use a HashSet?. Here the cost of adding and removing an element is much lower. And to solve the problem of removing a random element we could take an Iterator on the set, which iterates over the elements 'in no particular order'. Select the first element from the iterator and remove that from the set.

[bruno]

Regarding "iterates over the elements 'in no particular order'": I think this means you shouldn't rely on the ordering, including the fact this ordering would be useful as a random ordering. It just means that id's which hash to something at the start of the hashtable will be returned first by the iterator, and thus that the 'holes' will be created at the start of the map, while the filling of the hashmap will be spread out evenly over all the buckets.

I'm not convinced yet on the action to be taking when no record is found. I think the intention when one writes a test is that the actions will actually do something (the given number of times, or for the given duration). When this is not possible due to lacking data, it seems like a 'wrong' testscenario to me, and attention should be drown to this. This could be done by throwing an exception, or in case you just wait, by at least outputting big warnings so that this is visible to the user.