Take two at a water layer by ratmice · Pull Request #39 · osmus/layercake

ratmice · 2026-07-13T00:11:09Z

I went ahead and added comments to each of the tags I added.
I couldn't really understand how to do navigation without obstructions like dams/weir/waterfalls,
so I did end up including those, let me know if you really want those omitted.

Looking through the keys, they are pretty sparse for most of the regions I tried.
It seemed like one key would appear on a few items in one area. Ditto for a different key on a different region.
(Like open_water in the regions I tried only appeared in finland)

I'm not sure if you'll actually want map_from_tag_list for building a map from related tags.
It seemed weird to have separate columns for each vessel kind, for restricting access by kind of boat,
however the majority of these appear to have cardinality(vessel_access) == 1. So I'm not entirely sure it's the right thing to do.

jake-low

Sorry it took me a couple of weeks to reply to this. I've left a few comments.

I tend to be conservative about which tags to include in each layer, because it's easier to add a column than to remove it. OSM tags wax and wane over time, which works because of its fluid schema. But Layercake attempts to squish OSM into something rigid and typed, so removing a column later (e.g. if the community decides to change how something is tagged) is a breaking change for downstream users who reference that column in their SQL queries or similar. Most of my comments are from that perspective.

jake-low · 2026-07-30T01:57:12Z

+      -- Reason: dimensions.
+      tags['width']           AS width,
+      tags['depth']           AS depth,
+      tags['depth:']          AS 'depth:',


depth: looks like a typo?

Or at the very least should be a prefix_map, I'll try and figure out whether I was looking at a depth prefix, before just removing it.

I didn't see any of these in the data i've downloaded, but I believe that these were for the prefix_map mentioned in

https://wiki.openstreetmap.org/wiki/Key:depth

They prefix map seems used for things like how the depth tag was obtained/measured it's accuracy, etc.
depth:source_quality, depth:technique, depth:accuracy depth:exposition.

It may just be used more for water bodies than rivers?

jake-low · 2026-07-30T02:00:40Z

+      tags['height']          AS height,    -- For waterfalls?
+
+      -- Reason: navigation direction and obstruction bypass,
+      prefix_map('oneway:', tags)    AS 'oneway:',


I'd avoid using a trailing : in a column name; it's confusing. I also don't think we need to include oneway:* tags at all (yet) since there are only a handful of uses on waterways worldwide currently.

I suppose it doesn't matter if we're just removing the prefix_map, but is there any suggestion/convention for
tags that act as both a prefix_map and a tag? That's why I was using : here in column names.

The convention used in other layers is to pluralize the tag key. E.g. prefix_map('name:', tags) AS names. As a user you can then do SELECT name AS default_name, names['en'] AS english_name or similar which reads pretty well.

This is somewhat awkward for some tag keys though.

Ahh, yeah I think if I recall I might have started doing the : thing after some awkwardness.

tags['rapids'] as rapids, tags['rapids'] as rapidss,

seems the most likely culprit

One thought: prefix_map exists because name:* is an unbounded set of keys (every present and possible future language code, and more), so we can't enumerate it at build time. But rapids:* has a small, known set of keys. The best structure then is probably just to have the following fixed set of columns:

rapids

rapids:name

rapids:class

rapids:intermittent

(others can be added too but that's probably a good start)

This is how Layercake currently handles similar cases. For example building, building:material, and building:levels are each a top level column in the buildings layer.

An additional advantage of this is that top-level columns can be efficiently filtered or projected in a query; map columns cannot.

I couldn't find anything tagged with rapids:intermittent only two with rapids:class

mostly just rapids:name, plus 62 with rapids:description.

jake-low · 2026-07-30T02:00:42Z

+      -- Reason: navigation direction and obstruction bypass,
+      prefix_map('oneway:', tags)    AS 'oneway:',
+      tags['oneway']          AS oneway,
+      tags['canoe_pass']      AS canoe_pass,


canoe_pass doesn't look like an established tag (only 21 uses)

jake-low · 2026-07-30T02:01:33Z

+      tags['lock']            AS lock,
+
+      -- Boat access
+      map_from_tag_list(tags, ['motorboat', 'ship', 'sailboat', 'boat', 'canoe']) as vessel_access,


I think it's clearer to make each of these its own column (this is what the highways layer does).

jake-low · 2026-07-30T02:02:11Z

+      tags['hazard']          AS hazard,
+
+      -- Reason: Fish navigation?
+      tags['fish_pass']       AS fish_pass,   


fish_pass also doesn't look like an established tag (only 76 uses)

jake-low · 2026-07-30T02:05:18Z

+      -- Reason: Fish navigation?
+      tags['fish_pass']       AS fish_pass,   
+      -- Reason: hydrology
+      tags['order:strahler']  AS 'order:strahler',


I don't think this info belongs in OSM; Strahler number can easily be computed and is just one of several methods of analyzing stream topography. So I would not include it in Layercake.

jake-low · 2026-07-30T02:06:09Z

+      prefix_map('oneway:', tags)    AS 'oneway:',
+      tags['oneway']          AS oneway,
+      tags['canoe_pass']      AS canoe_pass,
+      tags['lock']            AS lock,


Might also want lock_name and lock_ref if we're including this.

jake-low · 2026-07-30T02:09:09Z

+    -- waterway-relevant tags i've added or changed...   
+      -- Reason: general
+      tags['usage']           AS usage,
+      tags['natural']         AS natural,


I'm unsure whether to include natural; I think most of the values come from a small set (valley, canyon, etc) and it's much more common for those features to be mapped as a separate element, so SELECT * FROM waterways WHERE natural = 'canyon' wouldn't actually be a reliable way to identify waterways that flow through canyons.

I think the exact reason I included is was because of https://wiki.openstreetmap.org/wiki/Waterways

If a stream starts as as spring, the node can be tagged with natural=spring.

But now that I look at that it, it's limited to nodes and areas, and this script has a where clause WHERE kind = 'line'

I'll just remove it for now.

jake-low · 2026-07-30T02:10:48Z

+    type,
+    id,
+    tags['waterway']        AS waterway,
+    tags['name']            AS name,


Probably want to split_mutli on name, and include prefix_map_split('names:') too, since major rivers that pass through or form the border between different countries will usually have multiple names.

jake-low · 2026-07-30T02:14:32Z

+    WHERE kind = 'line'
+      AND tags['waterway'] IN ('river', 'stream', 'canal', 'ditch', 'drain', 'flowline', 'fairway', 'link',
+        -- Reason: Routing obstructions.
+        'dam', 'weir', 'waterfall'


I'm mixed about whether including dams, weirs, and waterfalls is useful. We do not include barrier elements in the highways layer, which would be necessary for correct highway routing. And it seems like it might surprise data consumers that dams, weirs, and waterfalls are included in the "waterways" layer (IMO this is one of the many confusing things about OSM's taxonomy; natural=waterfall and man_made=dam would be clearer, but alas).

I'll just drop damns, weirs, and waterfalls for now, it can always be added later.

On man_made=dam it isn't really common but there are also natural=beaver_dam it's wiki page is actually hilarious.
https://wiki.openstreetmap.org/wiki/Tag:natural%3Dbeaver_dam

While similar to the man_made=, beaver_made='s use count was too low to be kept as good practice.

Since we removed that I also got rid of height which is probably only useful for waterfalls.

ratmice · 2026-07-30T02:53:29Z

Thank you for the review, I'll try and work on your suggestions tomorrow

ratmice · 2026-07-30T05:19:47Z

I went ahead and pushed some changes, plus some reformatting in a separate commit to make everything look nicer.

One question that I had, is this uses the general Geometry type of duckdb, even though it's doing a WHERE kind = line ,
I think, but am uncertain whether we could use a more specific gemetry type. I think it still needs to be the general Geometry because it could be either LineString or MultiLineString?

Is there a goal of optimizing the geometry for specific types in layercake?

ratmice · 2026-08-03T06:27:06Z

I guess one thing I wonder is if you want to just wait for another person who can use the layer in their program
(since mine needs more than the fields we've discussed), before committing to a set of fields. It probably makes more sense, just to know you're avoiding committing to a set of fields no-one is currently using?

jake-low · 2026-08-05T00:05:15Z

One question that I had, is this uses the general Geometry type of duckdb, even though it's doing a WHERE kind = line, I think, but am uncertain whether we could use a more specific geometry type.

DuckDB actually does this automatically when writing parquet files. It observes what types get written into the column and then sets the appropriate type annotation at the end. I checked and the geometry_type annotation in your current version of this layer is [LineString] since that's the only type that appears in it.

I'm not aware of any software that actually uses this info to optimize how it handles the data though, so it's mainly just advisory info that might appear when someone inspects the file.

I guess one thing I wonder is if you want to just wait for another person who can use the layer in their program (since mine needs more than the fields we've discussed), before committing to a set of fields.

I think it's okay to ship this even if we don't know exactly who will use it. There's lots of reasons someone might want to grab all of the waterways in OSM, and this layer makes that easy. The set of attributes we've arrived at feels like a reasonable starting point; we can always tweak it later.

If you mark this PR as "ready for review" I am happy to rebase it, review it, and test it.

ratmice · 2026-08-05T00:10:19Z

Okay I marked it as ready for review,
I am also happy to take care of the conflicts if that would be helpful.
Thank you for your reviews and attention to detail so far, happy to continue working on it if anything further needs doing.

Add a water layer

a23407a

jake-low reviewed Jul 30, 2026

View reviewed changes

ratmice added 2 commits July 29, 2026 22:12

Remove some tags after review

d22b7c0

remove extra indentation

ba5587f

ratmice marked this pull request as ready for review August 5, 2026 00:06

Uh oh!

Uh oh!

Conversation

ratmice commented Jul 13, 2026

Uh oh!

jake-low left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ratmice commented Jul 30, 2026

Uh oh!

ratmice commented Jul 30, 2026

Uh oh!

ratmice commented Aug 3, 2026

Uh oh!

jake-low commented Aug 5, 2026

Uh oh!

ratmice commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants