Timeline
2010-10-13:
- 15:36 Ticket #130 (IndexerWorker: start multiple listeners for each subscription) closed by
- fixed
- 15:35 Changeset [4378] by
- Augment number of listeners per index back to 3, works fine.
- 15:17 Changeset [4377] by
- Update outdated server process readme.
- 15:07 Changeset [4376] by
- IndexerMaster?: fix NPE in JobClient? instantiation (since move to hadoop …
- 14:49 Changeset [4375] by
- Upgrade to newer Kauri, remove duplication of kauri-related version …
- 14:12 Changeset [4374] by
- Make sure RowLogEndToEndTest? sets the amount of expected messages before …
- 13:53 Ticket #144 (RowLocker should use more complex token to identify lock ownership) created by
- Currently RowLocker? uses System.currentTimeMillis as token to identify who …
- 13:38 Changeset [4373] by
- Binary dist: some finishing touches.
- 12:55 Changeset [4372] by
- Various fixes to windows scripts. Use newer version of maven assembly …
- 12:37 Changeset [4371] by
- Correcting typo in log message.
- 12:22 Changeset [4370] by
- Let the RemoteListenerHandler? only write to the channel when the channel …
- 10:24 Changeset [4369] by
- Improved the way InterruptedException? is handled by the RowLog? and other …
- 10:08 Changeset [4368] by
- Add a maven plugin to download kauri-runtime-launcher and its dependencies …
- 09:46 Changeset [4367] by
- Binary distribution: adding Java service wrapper. Adding Windows variants …
- 09:36 Changeset [4366] by
- Handle border case
- 09:35 Changeset [4365] by
- Build system: remove requirement to download SOLR and to configure its …
- 06:55 Changeset [4364] by
- Improve error handling when publishing and unpublishing host information …
2010-10-12:
- 15:51 Changeset [4363] by
- Introducing binary distribution build: * Added a Maven plugin to produce …
- 15:29 Changeset [4362] by
- Make the RowLogEndToEndTests? stop when they take too long.
- 14:53 Changeset [4361] by
- RowLogTest? should also wait for the subscriptions to be known by the …
- 14:13 Changeset [4360] by
- Make sure when putting messages on the rowlogshard the same set of …
- 11:41 Changeset [4359] by
- Let the remoteTypeManager only initialize its cache once the AvroConnector? …
- 08:26 Changeset [4358] by
- row log end to end test: use a large ZooKeeper? timeout, seems to help the …
2010-10-11:
- 16:13 Changeset [4357] by
- Fix json parsing in RestTest? (should use non-std json features). Changed …
- 15:44 Changeset [4356] by
- Disabling RowLogLocalEndToEndTest? again. Still seems to give problems on …
- 15:29 Changeset [4355] by
- RemoteTypeManager? caches FieldTypes? and RecordTypes? and refreshes its …
- 14:21 Ticket #61 (Fully-distributed setup) closed by
- fixed: We can now run multiple Lily nodes, the important topics have been …
- 14:16 Ticket #33 (Handle disappearing and reappearing ZK connection in Lily nodes) closed by
- fixed: Closing this issue, ZooKeeper? usage and understanding should be much …
- 14:05 Ticket #74 (Routing towards different SOLR shards in the Indexer) closed by
- fixed
- 14:05 Ticket #70 (Indexer should lock record it is working on) closed by
- fixed: Considering this as ok for 0.2.
- 14:02 Ticket #133 (Implement all increment index update states) closed by
- fixed: (In [4354]) Indexer: implement removal of MQ subscription when state is …
- 14:02 Changeset [4354] by
- Indexer: implement removal of MQ subscription when state is …
- 13:58 Ticket #127 (Growing amount of Watchers on ZK by RowLogConfigurationManager) closed by
- fixed: This was fixed in r4330.
- 12:31 Changeset [4353] by
- SubscriptionExecutionState?: write a version number before the serialized …
- 12:23 Changeset [4352] by
- json/jackson use cleanup
- 11:31 Changeset [4351] by
- Implement JsonFormat?.deserialize also using ObjectMapper?. Just cleanup, …
- 11:29 Changeset [4350] by
- Declare version number of maven-avro-plugin in root pom.
- 09:58 Changeset [4349] by
- Changing HBase/Hadoop to the Cloudera versions. HBase is patched with the …
2010-10-09:
- 13:14 Changeset [4348] by
- indexer worker config: change default to 1 index updater per indexer (per …
- 12:56 Changeset [4347] by
- mbox-import tool: improve performance of MboxInputStream?
- 11:14 Changeset [4346] by
- Indexer engine: the Tika parser is thread safe and takes quite some time …
2010-10-08:
- 16:47 Changeset [4345] by
- SubscriptionExecutionState?: changing serialized format from JSON to Avro. …
- 16:43 Changeset [4344] by
- Add missing argument.
- 16:19 Changeset [4343] by
- mbox import tool: avoid the update of the message record by letting the …
- 16:10 Changeset [4342] by
- Let HBaseTypeManager.getFieldTypeById and getRecordTypeById make use of …
- 15:52 Changeset [4341] by
- Adding dependency on lily-zk-util.
- 14:52 Changeset [4340] by
- TypeManager? uses Zookeeper to inform and get informed of any changes so …
- 13:31 Changeset [4339] by
- Improve Jackson usage: do not construct an ObjectMapper? upon each use, but …
- 13:07 Changeset [4338] by
- IndexerWorker?: start multiple listeners for each index updater, re #130.
- 12:20 Ticket #143 (Renaming LilyCMS) created by
- The overall project is now called 'Lily Project', where (previously) …
- 09:44 Ticket #135 (IndexerWorker: review for correct ZK usage) closed by
- fixed: IndexerWorker? does not shut down its index updaters when the ZK connection …
- 09:40 Ticket #137 (IndexerModel: review for correct ZK usage) closed by
- fixed: IndexerModel? had quite some problems in its ZK use, it was improved in …
- 09:38 Ticket #134 (ZkUtil.retryOperation implementation improvement) closed by
- fixed: Implemented in r4325.
- 09:37 Ticket #136 (ZooKeeperItf: add method to add default watchers) closed by
- fixed: Implemented in r4325.
- 09:36 Ticket #139 (IndexerMaster: assure assigned subscription IDs are unique) closed by
- fixed: The subscription ID's for the index updater are now prefixed with …
- 09:35 Changeset [4337] by
- Change prefix for index update MQ subscriptions from 'IndexUpdater?: ' to …
- 09:26 Changeset [4336] by
- RowLogConfigurationManager?.addSubscription: remove …
- 09:05 Changeset [4335] by
- Forgot to add this file in r4333.
- 08:44 Ticket #119 (Consider renaming SubscriptionContext to RowLogSubscription) closed by
- fixed: (In [4334]) Renamed Subscription to RowLogSubscription? as I see that is …
- 08:44 Changeset [4334] by
- Renamed Subscription to RowLogSubscription? as I see that is what was …
- 08:40 Changeset [4333] by
- Store subscription data as JSON instead of simple byte concatenation, will …
2010-10-07:
- 16:52 Changeset [4332] by
- Rowlog conf mgr: some initial work on exception handling and introducing …
- 16:51 Changeset [4331] by
- IndexerModelTest?: do not rely on exact ordering of events (depends on …
- 15:57 Changeset [4330] by
- Improved ZooKeeper? usage in rowlog: made it safe against connection …
- 12:13 Ticket #24 (Optimize loading of versioned fields) closed by
- fixed: r4329 We went for solution 1. But instead of using timestamp 0, we use …
- 12:11 Changeset [4329] by
- Optimize loading of versioned fields. (#24) Deleting a record just marks …
- 09:28 Changeset [4328] by
- Fix and simplify ZK watcher within IndexerModelImpl?: we made use of …
2010-10-06:
- 13:08 Changeset [4327] by
- Since HBase mini cluster shutdown has a tendency of sometimes failing …
- 12:37 Changeset [4326] by
- Keep track of what the zookeeper event thread is, and throw an exception …
- 11:50 Changeset [4325] by
- Move retryOperation from ZkUtil? to ZooKeeperItf?. Added methods for …
- 11:50 Changeset [4324] by
- Simplify: a complete method wrapped in synchronized(this) can as well be …
- 10:33 Changeset [4323] by
- RowLogProcessorImpl?: fix a possible deadlock: subscriptionsChanged() is …
- 09:57 Changeset [4322] by
- Learned a nice trick to work around the lack of an IdentitySet?: …
- 09:39 Changeset [4321] by
- Rename ZkUtil?.retryOperationForever method to ZkUtil?.retryOperation. This …
- 09:34 Changeset [4320] by
- Make sure the ZK client connection is closed when shutting down the Lily …
- 09:31 Ticket #129 (Upgrade to Avro 1.4) closed by
- fixed: Did the upgrade of Avro in r4319. Did not yet use Netty connector, simply …
- 09:07 Changeset [4319] by
- Upgrade to more recent Avro. Was going to use the 1.4.0 release but since …
- 09:00 Ticket #142 (LilyClient should be able to deal with bad connections) created by
- LilyClient? should have some way to deal with repositories which are in its …
2010-10-05:
- 14:50 Ticket #63 (Make server process Kauri modules for the RowLog MQ and WAL processors) closed by
- fixed: Implemented in r4308.
- 14:49 Ticket #56 (Run only one RowLogProcessor per shard cluster wide) closed by
- fixed: Leader election for running the message queue row log processor was added …
- 14:43 Ticket #141 (Add testcase for indexer components (master/worker/fullbuild)) created by
- We should have an automated testcase for these components, to verify at …
- 14:40 Ticket #140 (Review indexer engine code for not eating InterruptedExceptions) created by
- One case I've noticed is VersionTag? which catches Throwables. This is …
- 14:38 Ticket #139 (IndexerMaster: assure assigned subscription IDs are unique) created by
- Right now the IndexerMaster? is not sure that the MQ subscription it …
- 14:35 Ticket #138 (Re-activate RowLogLocalEndToEndTest) created by
- This test is currently completely commented out since it was failing, see …
- 14:33 Ticket #137 (IndexerModel: review for correct ZK usage) created by
- IndexerModel? needs some review for its ZK usage. Especially not eating …
- 14:31 Ticket #136 (ZooKeeperItf: add method to add default watchers) created by
- Especially for watching connection-related state events, it is interesting …
- 14:29 Ticket #135 (IndexerWorker: review for correct ZK usage) created by
- The IndexerWorker? needs to be review for correct ZK usage, especially with …
- 14:28 Ticket #134 (ZkUtil.retryOperation implementation improvement) created by
- The current implementation of ZkUtil?.retryOperation is not very …
- 14:25 Ticket #133 (Implement all increment index update states) created by
- The index update states SUBSCRIBE_DO_NOT_LISTEN and DO_NOT_SUBSCRIBE still …
- 14:25 Ticket #132 (Error handling in indexer fullbuild map reduce job) created by
- We need to have a look at error handling in the fullbuild MR job. …
- 14:22 Ticket #131 (Parallel processing in the fullbuild mapreduce job) created by
- For similar reasons as in #130, the fullbuild mapreduce job should run …
- 14:21 Ticket #130 (IndexerWorker: start multiple listeners for each subscription) created by
- This should improve indexing throughput, which spends quite some time …
- 14:20 Ticket #129 (Upgrade to Avro 1.4) created by
- This is in the context of a wrapping of ConnectException? withing an …
- 13:34 Ticket #88 (Clean shutdown of the Lily node) closed by
- fixed
- 13:27 Ticket #128 (Consistent approach for creating HBase schema) created by
- 09:41 Ticket #127 (Growing amount of Watchers on ZK by RowLogConfigurationManager) created by
- Stopping and starting the RowLogProcessor? results in a growing amount of …
- 09:27 Changeset [4318] by
- Server process / REST itf: do not use a fixed port number for the HTTP …
- 09:26 Ticket #126 (Add logging to RowLog) created by
- Add more informative logging to the RowLog? framework.
- 09:21 Ticket #125 (Refine RowLogMessageListenerMapping to listener level) created by
- Currently the RowLogMessageListenerMapping? maps SubscriptionIds? to …
- 09:07 Ticket #124 (Use stateless, singleton ChannelHandlers in ...) created by
- Let the RemoteListenersSubscriptionHandler? use stateless, singleton …
- 09:06 Changeset [4317] by
- Server process: activate the message queue row log processor on only 1 …
- 09:01 Ticket #123 (Create existing subscription should inform caller) created by
- When calling the addSubscription method on RowLogConfigurationManager? it …
- 08:57 Ticket #122 (Update subscriptions on the RowLogConfigurationManager) created by
- Add some calls to allow to update existing subscriptions on the …
- 08:55 Ticket #121 (Extend RowLogConfigurationManager for RowLog) created by
- Extend the RowLogConfigurationManager? with calls for configuration of the …
- 08:40 Ticket #120 (ListenerId is used for either an id, or a host+port) created by
- The RowLogConfigurationManager#addListener? call accepts a string as …
- 08:19 Ticket #119 (Consider renaming SubscriptionContext to RowLogSubscription) created by
- The SubscriptionContext? is a value object describing the subscription that …
- 08:11 Ticket #118 (Review error handling in RowLog) created by
- Review the errors that are thrown in and by the RowLog? and related …
- 07:52 Ticket #27 (Introduce AbstractRepository) closed by
- wontfix: An AbstractRepository? class is not really needed here. It is ok like it is …
2010-10-04:
- 16:07 Changeset [4316] by
- Some test cleanup/fixes which makes the tests run for me without forkMode …
- 16:01 Changeset [4315] by
- Remove empty dir.
- 13:34 Changeset [4314] by
- Enable link index updating in the server process: * Moved …
- 13:03 Changeset [4313] by
- Move rowlog-impl dependency from indexer-engine to indexer-worker, since …
- 12:56 Changeset [4312] by
- Clean up remaining getId()/getMaxTries() methods in RowLogMessageListener? …
- 12:38 Changeset [4311] by
- Indexer fullbuild MR job: - pass a dummy rowlog impl to the embedded …
- 12:19 Changeset [4310] by
- Google Guava became a HBase client dependency.
- 09:50 Ticket #117 (Test Lily against CDH-current) created by
- Lily must install and run well against Cloudera Distribution for Hadoop …
2010-10-01:
- 16:21 Changeset [4309] by
- * RowLogLocalEndToEndTest?: temporarily commented out the whole test as it …
- 15:17 Ticket #64 (Pushing messages to listeners in other JVMs) closed by
- fixed: The RowLogConfigurationManager? allows to define if the listeners of a …
- 14:55 Ticket #116 (Allow for a mix of local and remote RowLog listeners on a subscription) created by
- The RowLogConfigurationManager? allows to define at subscription level if …
- 14:49 Ticket #49 (MQ should serve multiple listeners) closed by
- fixed: The RowLogConfigurationManager? allows to register multiple listeners on a …
- 14:29 Ticket #66 (Notify RowLog and RowLogProcessor when consumers are (un-)registered) closed by
- fixed: The RowLogConfigurationManager? allows to register subscriptions (formerly …
- 14:26 Ticket #52 (RowLog should not call the RowLogProcessor directly when new consumers are ...) closed by
- fixed: The RowLogConfigurationManager? allows to register subscriptions (formerly …
- 14:10 Changeset [4308] by
- Adjust the Lily server node to the changes happening in rowlog-land: * …
- 13:46 Changeset [4307] by
- Move to latest HBase trunk with patch for HBASE-3064 (Long sleeping in …
- 13:30 Changeset [4306] by
- RowLogConfigurationManager? should not close the zookeeper connection it …
- 12:34 Changeset [4305] by
- RowLogConfigurationManager? now accepts a ZooKeeperItf? as parameter in its …
- 11:14 Changeset [4304] by
- Move MessageQueueFeeder? from repository-impl to rowlog-impl project, …
- 09:58 Changeset [4303] by
- Pull Wal (RowLog?) instantiation out of HBaseRepository
- 09:49 Changeset [4302] by
- Improvements to ZooKeeper? usage within LilyClient?: - check on startup if …
- 08:38 Ticket #62 (Pull RowLog processor instantiation out of Repository) closed by
- fixed: r4297
- 08:19 Changeset [4301] by
- Rename ListenerClassMapping? to RowLogMessageListenerMapping?. Register …
2010-09-30:
- 16:06 Ticket #111 (Write ahead log secondary actions (= rowlog consumers) error handling) closed by
- fixed: Subscriptions on a Rowlog can be ordered by giving them an order number. A …
- 16:04 Changeset [4300] by
- Subscriptions on a Rowlog can be ordered by giving them an order number. A …
2010-09-29:
- 15:53 Changeset [4299] by
- Refactored the IndexerMaster? to better handle ZK disconnection/connection …
- 15:27 Changeset [4298] by
- Also setup messageQueue and messageQueueProcessor in AvroRepositoryTest?
- 14:51 Changeset [4297] by
- RowLog? subscriptions and listeners are configured through the …
- 14:03 Changeset [4296] by
- Fix indexing of deref values pointing to multivalue fields (needs …
- 10:05 Changeset [4295] by
- Publish an MBean for ZooKeeper? so that one can query what the ZK session …
2010-09-28:
- 16:19 Changeset [4294] by
- ZooKeeper?: * use either ZkUtil?.connect or StateWatchingZooKeeper?, not …
- 14:19 Changeset [4293] by
- ZkUtil?: removed the ZkPathCreationException? which was only used to wrap …
- 14:05 Changeset [4292] by
- Let ZkPublisher? reuse the global ZooKeeper? instance instead of …
- 13:34 Changeset [4291] by
- ZooKeeper? (re #33): - wait for connection to come up at start up (this is …
- 07:43 Changeset [4290] by
- ZooKeeper? related stuff: * IndexLocker?: handle unnoticed creation of the …
2010-09-27:
- 10:11 Changeset [4289] by
- ZkLock? + LeaderElection?: improve / fix behavior in case of connection …
- 07:55 Changeset [4288] by
- Renaming and refactoring some RowLog? classes.
2010-09-24:
- 14:08 Changeset [4287] by
- Lily node HTTP port: switch from NIO to BIO Jetty connector, re #107.
2010-09-23:
- 13:55 Changeset [4286] by
- mbox-import readme: list some more mbox sources
- 07:51 Changeset [4285] by
- Indexer engine: test classes/resources were still in old package, moving …
2010-09-22:
- 16:57 Changeset [4284] by
- Avoid creating too many workers and futures when there are no messages to …
- 16:20 Changeset [4283] by
- Introduce the record index lock (re #70). To avoid multiple parallel …
- 16:17 Ticket #115 (Indexer: handle record messages (events) in order) created by
- One of the reasons for designing the message queue (rowlog) with a 'local …
- 15:24 Changeset [4282] by
- Move connecting to ZooKeeper? from RowLogConfigurationManager? to ZkUtil?
- 12:08 Changeset [4281] by
- Avoid putting messages in the worker queue that are being processed at the …
- 10:13 Changeset [4280] by
- Add possibility to delete indexes: - if a batch build job is currently …
2010-09-20:
- 16:33 Changeset [4279] by
- Giving RowLogConfigurationManager? more time to connect to Zookeeper
- 16:23 Changeset [4278] by
- Added basic framework for using multiple remote listeners for one rowlog …
- 16:02 Changeset [4277] by
- Tests: enabling forkMode always, just to see if this would help with the …
- 13:22 Changeset [4276] by
- KauriTestUtility?: generate hbase.xml in new format, otherwise it has no …
- 12:30 Changeset [4275] by
- Upgrade to newer HBase trunk version.
- 12:21 Ticket #114 (HDFS blob storage: consider hierarchical directory layout) created by
- Currently all HDFS-stored blobs are stored within a single directory on …
- 12:15 Changeset [4274] by
- Repository: dfs blob store: store blobs in a specific directory rather …
- 10:07 Changeset [4273] by
- Allows to build these plugin-dependencies without the root-project being …
- 07:52 Changeset [4272] by
- Remove a temporary system.out.println
2010-09-16:
- 16:31 Changeset [4271] by
- Indexing MR job: skip deleted records when scanning, use actual HBase …
- 16:13 Changeset [4270] by
- Indexer admin cli tools: check that the specified SOLR shard URIs are not …
- 15:50 Changeset [4269] by
- Server process: align the hbase.xml conf with the mapreduce.xml one: this …
- 15:29 Changeset [4268] by
- Add indexer admin CLI tools to retrieve the indexerconf and shardingconf …
- 15:03 Changeset [4267] by
- Set svn:ignore props.
- 15:01 Changeset [4266] by
- LilyClient? fix: blobs were not written to the real DFS because the DFS URI …
- 14:23 Changeset [4265] by
- Bring some uniformity into the CLI tools and avoid code duplication …
- 12:48 Changeset [4264] by
- * Moved indexerconf from indexer-engine to indexer-model project. This …
- 09:33 Changeset [4263] by
- Indexerconf: move the active behavior, more specificaly the Value.eval() …
2010-09-15:
- 16:38 Changeset [4262] by
- ShardSelectorTest?: fix wrong paths to test files.
- 16:10 Changeset [4261] by
- HbaseProxy?: make sure HBaseTestingUtility uses a new datadir on each run.
- 14:55 Changeset [4260] by
- Indexer: introducing SOLR sharding. If you specify multiple SOLR shards …
- 14:13 Changeset [4259] by
- mbox import: fix error in schema, add some ideas for future improvements …
2010-09-14:
- 11:43 Changeset [4258] by
- Making the mbox import tool a bit more useful: - support importing .gz …
- 11:40 Ticket #113 (Indexer: handle blob extraction errors.) created by
- This is a specific case of #76 (indexer error handling). Some relation to …
- 09:35 Changeset [4257] by
- Uncomment indexer tests. Version tag 'last' now has special behaviour, …
- 09:32 Changeset [4256] by
- Upgrade to latest HBase.
2010-09-13:
- 16:43 Changeset [4255] by
- Add a tool for importing mbox mail archives into Lily. The purpose is to …
- 16:27 Changeset [4254] by
- Indexer: fix problem with tika not finding its parsers when running as …
Note: See TracTimeline
for information about the timeline view.