Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/dirlist.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@ To begin, ingest some data with Ingest.java.
$ ./bin/runex dirlist.Ingest --vis exampleVis --chunkSize 100000 /local/username/workspace

This may take some time if there are large files in the /local/username/workspace directory. If you use 0 instead of 100000 as the `chunkSize`, the ingest will run much faster, but it will not put any file data into Accumulo (the dataTable will be empty).
Note that running this example will create tables `examples.dirTable`, `examples.indexTable`, and `examples.dataTable` in Accumulo that you should delete when you have completed the example.
Note that running this example will create tables `examples.dirTable`, `examples.indexTable`, and `examples.dataTable` in Accumulo that you should delete when you have completed this example (dirlist.md) and the File System Archive Example Data Only (filedata.md).
If you modify a file or add new files in the directory ingested (e.g. /local/username/workspace), you can run Ingest again to add new information into the Accumulo tables.

To browse the data ingested, use Viewer.java. Be sure to give the "username" user the authorizations to see the data (in this case, run

$ accumulo shell -u root -e 'setauths -u username -s exampleVis'
$ accumulo shell -u root -e 'setauths -u root -s exampleVis'

then run the Viewer:

Expand Down
6 changes: 4 additions & 2 deletions docs/helloworld.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,13 @@ Inserts data with a BatchWriter:

$ ./bin/runex helloworld.Insert

On the accumulo status page at the URL below (you may need to replace 'localhost' with the name or IP of your server), you should see 50K entries
Open the Accumulo Monitor, navigate to the `examples.hellotable` table, and
verify that it shows 50K entries. You may need to replace `localhost` with the
name or IP address of your server.

http://localhost:9995/

To view the entries, use the shell (run `accumulo shell -u username -p password` to access it) to scan the table:
To view the entries, use the shell (run `accumulo shell --user root --password secret` to access it) to scan the table:

username@instance> table examples.hellotable
username@instance examples.hellotable> scan
Expand Down
2 changes: 1 addition & 1 deletion docs/isolation.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Below, Interference Test is run without isolation enabled for 5000 iterations
and it reports problems.


$ accumulo shell -u <username> -p <password> -e 'createnamespace examples'
$ accumulo shell --user root --password secret --execute-command 'createnamespace examples'
$ ./bin/runex isolation.InterferenceTest -t examples.isotest --iterations 50000
ERROR Columns in row 053 had multiple values [53, 4553]
ERROR Columns in row 061 had multiple values [561, 61]
Expand Down
19 changes: 10 additions & 9 deletions docs/sample.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,22 +95,23 @@ The commands below look under the hood to give some insight into how this
feature works. The commands determine what files the sampex table is using.

root@instance> tables -l
accumulo.fate => +fate
accumulo.metadata => !0
accumulo.replication => +rep
accumulo.root => +r
accumulo.scanref => +scanref
examples.sampex => 2
trace => 1
root@instance sampex> scan -t accumulo.metadata -c file -b 2 -e 2<
2< file:hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf [] 702,8
2< file:{"path":"hdfs://localhost:8020/accumulo/tables/2/default_tablet/A000000s.rf","startRow":"","endRow":""} [] 702,8
2< file:{"path":"hdfs://localhost:8020/accumulo/tables/2/default_tablet/F000009w.rf","startRow":"","endRow":""} [] 439,1

Below shows running `accumulo rfile-info` on the file above. This shows the
Below shows running `accumulo file rfile-info` on the file above. This shows the
rfile has a normal default locality group and a sample default locality group.
The output also shows the configuration used to create the sample locality
group. The sample configuration within a rfile must match the tables sample
configuration for sample scan to work.

$ accumulo rfile-info hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf
Reading file: hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf
$ accumulo file rfile-info hdfs://localhost:8020/accumulo/tables/2/default_tablet/A000000s.rf
Reading file: hdfs://localhost:8020/accumulo/tables/2/default_tablet/A000000s.rf
RFile Version : 8

Locality group : <DEFAULT>
Expand Down Expand Up @@ -170,15 +171,15 @@ After enabling sampling, the command below counts the number of documents in
the sample containing the words `import` and `int`.

$ ./bin/runex shard.Query --sample -t examples.shard import int | fgrep '.java' | wc
4 4 395
4 4 422

The command below counts the total number of documents containing the words
`import` and `int`.

$ ./bin/runex shard.Query -t examples.shard import int | fgrep '.java' | wc
382 382 40084
409 409 45726

The counts 4 out of 395 total are around what would be expected for a modulus
The counts 4 out of 422 total are around what would be expected for a modulus
of 101. Querying the sample first provides a quick way to estimate how much data
the real query will bring back.

Expand Down
23 changes: 13 additions & 10 deletions docs/shard.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,21 +37,24 @@ After creating the tables, index some files. The following command indexes all t
The following command queries the index to find all files containing 'foo' and 'bar'.

$ ./bin/runex shard.Query -t examples.shard foo bar
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/BaseHostRegexTableLoadBalancerTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerReconfigurationTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/TransformingIteratorTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/conf/PropertyTypeTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/WholeRowIteratorTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iteratorsImpl/IteratorConfigUtilTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/VisibilityEvaluatorTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/data/KeyBuilderTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerReconfigurationTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/ColumnVisibilityTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/clientImpl/TableOperationsHelperTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/conf/PropertyTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/admin/NewTableConfigurationTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/summary/SummaryCollectionTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/IteratorSettingTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/data/KeyExtentTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/VisibilityEvaluatorTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/TransformingIteratorTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/admin/NewTableConfigurationTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/compaction/RatioBasedCompactionPlannerTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/client/IteratorSettingTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/HostRegexTableLoadBalancerTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/conf/HadoopCredentialProviderTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/clientImpl/TableOperationsHelperTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/spi/balancer/BaseHostRegexTableLoadBalancerTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iteratorsImpl/IteratorConfigUtilTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/security/ColumnVisibilityTest.java
/path/to/accumulo/core/src/test/java/org/apache/accumulo/core/iterators/user/WholeColumnFamilyIteratorTest.java

In order to run ContinuousQuery, we need to run Reverse.java to populate the `examples.doc2term` table.
Expand Down
15 changes: 14 additions & 1 deletion src/main/java/org/apache/accumulo/examples/dirlist/Viewer.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
package org.apache.accumulo.examples.dirlist;

import java.awt.BorderLayout;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.util.Enumeration;
import java.util.Map;
Expand Down Expand Up @@ -210,10 +212,21 @@ static class Opts extends QueryUtil.Opts {
public static void main(String[] args) throws Exception {
Opts opts = new Opts();
opts.parseArgs(Viewer.class.getName(), args);
try (AccumuloClient client = opts.createAccumuloClient()) {
AccumuloClient client = opts.createAccumuloClient();
try {
Viewer v = new Viewer(client, opts);
v.setDefaultCloseOperation(DISPOSE_ON_CLOSE);
v.addWindowListener(new WindowAdapter() {
@Override
public void windowClosed(WindowEvent e) {
client.close();
}
});
v.init();
v.setVisible(true);
} catch (Exception e) {
client.close();
throw e;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ public static void main(String[] args)
}
}
}
client.tableOperations().flush(HELLO_TABLE, null, null, true);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@
*/
public class SampleExample {

// a compaction strategy that only selects files for compaction that have no sample data or sample
// data created in a different way than the tables
// A compaction selector that only selects files with no sample data or sample data created with
// a different configuration than the table.
static final PluginConfig selectorCfg = new PluginConfig(
"org.apache.accumulo.tserver.compaction.strategies.ConfigurableCompactionStrategy",
"org.apache.accumulo.core.compaction.ShellCompactCommandSelector",
Map.of("SF_NO_SAMPLE", ""));
static final CompactionConfig NO_SAMPLE_STRATEGY = new CompactionConfig()
.setSelector(selectorCfg);
Expand Down
Loading