diff --git a/docs/dirlist.md b/docs/dirlist.md index 720c96c..9f80e90 100644 --- a/docs/dirlist.md +++ b/docs/dirlist.md @@ -34,12 +34,12 @@ To begin, ingest some data with Ingest.java. $ ./bin/runex dirlist.Ingest --vis exampleVis --chunkSize 100000 /local/username/workspace This may take some time if there are large files in the /local/username/workspace directory. If you use 0 instead of 100000 as the `chunkSize`, the ingest will run much faster, but it will not put any file data into Accumulo (the dataTable will be empty). -Note that running this example will create tables `examples.dirTable`, `examples.indexTable`, and `examples.dataTable` in Accumulo that you should delete when you have completed the example. +Note that running this example will create tables `examples.dirTable`, `examples.indexTable`, and `examples.dataTable` in Accumulo that you should delete when you have completed this example (dirlist.md) and the File System Archive Example Data Only (filedata.md). If you modify a file or add new files in the directory ingested (e.g. /local/username/workspace), you can run Ingest again to add new information into the Accumulo tables. To browse the data ingested, use Viewer.java. Be sure to give the "username" user the authorizations to see the data (in this case, run - $ accumulo shell -u root -e 'setauths -u username -s exampleVis' + $ accumulo shell -u root -e 'setauths -u root -s exampleVis' then run the Viewer: diff --git a/docs/helloworld.md b/docs/helloworld.md index 88e8679..da52171 100644 --- a/docs/helloworld.md +++ b/docs/helloworld.md @@ -25,11 +25,13 @@ Inserts data with a BatchWriter: $ ./bin/runex helloworld.Insert -On the accumulo status page at the URL below (you may need to replace 'localhost' with the name or IP of your server), you should see 50K entries +Open the Accumulo Monitor, navigate to the `examples.hellotable` table, and +verify that it shows 50K entries. You may need to replace `localhost` with the +name or IP address of your server. http://localhost:9995/ -To view the entries, use the shell (run `accumulo shell -u username -p password` to access it) to scan the table: +To view the entries, use the shell (run `accumulo shell --user root --password secret` to access it) to scan the table: username@instance> table examples.hellotable username@instance examples.hellotable> scan diff --git a/docs/isolation.md b/docs/isolation.md index 412478a..67927e5 100644 --- a/docs/isolation.md +++ b/docs/isolation.md @@ -31,7 +31,7 @@ Below, Interference Test is run without isolation enabled for 5000 iterations and it reports problems. - $ accumulo shell -u -p -e 'createnamespace examples' + $ accumulo shell --user root --password secret --execute-command 'createnamespace examples' $ ./bin/runex isolation.InterferenceTest -t examples.isotest --iterations 50000 ERROR Columns in row 053 had multiple values [53, 4553] ERROR Columns in row 061 had multiple values [561, 61] diff --git a/docs/sample.md b/docs/sample.md index 05ac19f..3ad1ca0 100644 --- a/docs/sample.md +++ b/docs/sample.md @@ -95,22 +95,23 @@ The commands below look under the hood to give some insight into how this feature works. The commands determine what files the sampex table is using. root@instance> tables -l + accumulo.fate => +fate accumulo.metadata => !0 - accumulo.replication => +rep accumulo.root => +r + accumulo.scanref => +scanref examples.sampex => 2 - trace => 1 root@instance sampex> scan -t accumulo.metadata -c file -b 2 -e 2< - 2< file:hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf [] 702,8 + 2< file:{"path":"hdfs://localhost:8020/accumulo/tables/2/default_tablet/A000000s.rf","startRow":"","endRow":""} [] 702,8 + 2< file:{"path":"hdfs://localhost:8020/accumulo/tables/2/default_tablet/F000009w.rf","startRow":"","endRow":""} [] 439,1 -Below shows running `accumulo rfile-info` on the file above. This shows the +Below shows running `accumulo file rfile-info` on the file above. This shows the rfile has a normal default locality group and a sample default locality group. The output also shows the configuration used to create the sample locality group. The sample configuration within a rfile must match the tables sample configuration for sample scan to work. - $ accumulo rfile-info hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf - Reading file: hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf + $ accumulo file rfile-info hdfs://localhost:8020/accumulo/tables/2/default_tablet/A000000s.rf + Reading file: hdfs://localhost:8020/accumulo/tables/2/default_tablet/A000000s.rf RFile Version : 8 Locality group : @@ -170,15 +171,15 @@ After enabling sampling, the command below counts the number of documents in the sample containing the words `import` and `int`. $ ./bin/runex shard.Query --sample -t examples.shard import int | fgrep '.java' | wc - 4 4 395 + 4 4 422 The command below counts the total number of documents containing the words `import` and `int`. $ ./bin/runex shard.Query -t examples.shard import int | fgrep '.java' | wc - 382 382 40084 + 409 409 45726 -The counts 4 out of 395 total are around what would be expected for a modulus +The counts 4 out of 422 total are around what would be expected for a modulus of 101. Querying the sample first provides a quick way to estimate how much data the real query will bring back. diff --git a/docs/shard.md b/docs/shard.md index 05beea3..ac47ad5 100644 --- a/docs/shard.md +++ b/docs/shard.md @@ -37,21 +37,24 @@ After creating the tables, index some files. The following command indexes all t The following command queries the index to find all files containing 'foo' and 'bar'. $ ./bin/runex shard.Query -t examples.shard foo bar - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/BaseHostRegexTableLoadBalancerTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerReconfigurationTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/TransformingIteratorTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/conf/PropertyTypeTest.java /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/WholeRowIteratorTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iteratorsImpl/IteratorConfigUtilTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/VisibilityEvaluatorTest.java /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/data/KeyBuilderTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerReconfigurationTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/ColumnVisibilityTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/clientImpl/TableOperationsHelperTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/conf/PropertyTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/admin/NewTableConfigurationTest.java /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/summary/SummaryCollectionTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/IteratorSettingTest.java /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/data/KeyExtentTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/VisibilityEvaluatorTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/TransformingIteratorTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/admin/NewTableConfigurationTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/compaction/RatioBasedCompactionPlannerTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/IteratorSettingTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerTest.java /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/conf/HadoopCredentialProviderTest.java - /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/clientImpl/TableOperationsHelperTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/BaseHostRegexTableLoadBalancerTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iteratorsImpl/IteratorConfigUtilTest.java + /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/ColumnVisibilityTest.java /path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/WholeColumnFamilyIteratorTest.java In order to run ContinuousQuery, we need to run Reverse.java to populate the `examples.doc2term` table. diff --git a/src/main/java/org/apache/accumulo/examples/dirlist/Viewer.java b/src/main/java/org/apache/accumulo/examples/dirlist/Viewer.java index 3d8d1b3..592f74b 100644 --- a/src/main/java/org/apache/accumulo/examples/dirlist/Viewer.java +++ b/src/main/java/org/apache/accumulo/examples/dirlist/Viewer.java @@ -17,6 +17,8 @@ package org.apache.accumulo.examples.dirlist; import java.awt.BorderLayout; +import java.awt.event.WindowAdapter; +import java.awt.event.WindowEvent; import java.io.IOException; import java.util.Enumeration; import java.util.Map; @@ -210,10 +212,21 @@ static class Opts extends QueryUtil.Opts { public static void main(String[] args) throws Exception { Opts opts = new Opts(); opts.parseArgs(Viewer.class.getName(), args); - try (AccumuloClient client = opts.createAccumuloClient()) { + AccumuloClient client = opts.createAccumuloClient(); + try { Viewer v = new Viewer(client, opts); + v.setDefaultCloseOperation(DISPOSE_ON_CLOSE); + v.addWindowListener(new WindowAdapter() { + @Override + public void windowClosed(WindowEvent e) { + client.close(); + } + }); v.init(); v.setVisible(true); + } catch (Exception e) { + client.close(); + throw e; } } } diff --git a/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java b/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java index a7884be..139ee10 100644 --- a/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java +++ b/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java @@ -60,6 +60,7 @@ public static void main(String[] args) } } } + client.tableOperations().flush(HELLO_TABLE, null, null, true); } } } diff --git a/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java b/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java index 6b5e3c8..84da3f9 100644 --- a/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java +++ b/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java @@ -47,10 +47,10 @@ */ public class SampleExample { - // a compaction strategy that only selects files for compaction that have no sample data or sample - // data created in a different way than the tables + // A compaction selector that only selects files with no sample data or sample data created with + // a different configuration than the table. static final PluginConfig selectorCfg = new PluginConfig( - "org.apache.accumulo.tserver.compaction.strategies.ConfigurableCompactionStrategy", + "org.apache.accumulo.core.compaction.ShellCompactCommandSelector", Map.of("SF_NO_SAMPLE", "")); static final CompactionConfig NO_SAMPLE_STRATEGY = new CompactionConfig() .setSelector(selectorCfg);