-
Notifications
You must be signed in to change notification settings - Fork 10
feat: vendor Mustache templating engine & add Interpolator (AIC-2695) #172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ctawiah
wants to merge
4
commits into
feat/AIC-2662/ai-sdk-datamodel
Choose a base branch
from
feat/AIC-2695/ai-sdk-vendor-mustache
base: feat/AIC-2662/ai-sdk-datamodel
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
74b5692
feat: vendor Mustache templating engine & add Interpolator (AIC-2695)
ctawiah 97d46c1
refactor: encode LDContext into ldctx map directly (AIC-2695)
ctawiah 8969c72
docs: generalize cross-SDK interpolation policy wording (AIC-2695)
ctawiah 04326f0
fix: omit kind on nested per-kind ldctx entries for multi-kind contex…
ctawiah File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| This product includes vendored third-party source code. The relevant licenses | ||
| and copyright notices are reproduced below. | ||
|
|
||
| ================================================================================ | ||
| JMustache (jmustache) | ||
| -------------------------------------------------------------------------------- | ||
| Vendored from: com.samskivert:jmustache:1.15 | ||
| Upstream: https://github.com/samskivert/jmustache | ||
| Location: src/main/java/com/launchdarkly/sdk/server/ai/internal/mustache/ | ||
|
|
||
| The JMustache source has been relocated into the internal, non-public package | ||
| com.launchdarkly.sdk.server.ai.internal.mustache and is compiled from source as | ||
| part of this library. Aside from the relocated package declaration and a short | ||
| provenance banner at the top of each file, the source is unmodified from the | ||
| upstream 1.15 release. | ||
|
|
||
| License: The (New) BSD License (BSD 3-Clause) | ||
|
|
||
| Copyright (c) 2010, Michael Bayne | ||
| All rights reserved. | ||
|
|
||
| Redistribution and use in source and binary forms, with or without | ||
| modification, are permitted provided that the following conditions are met: | ||
|
|
||
| * Redistributions of source code must retain the above copyright notice, | ||
| this list of conditions and the following disclaimer. | ||
| * Redistributions in binary form must reproduce the above copyright notice, | ||
| this list of conditions and the following disclaimer in the documentation | ||
| and/or other materials provided with the distribution. | ||
| * The name Michael Bayne may not be used to endorse or promote products | ||
| derived from this software without specific prior written permission. | ||
|
|
||
| THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND | ||
| ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
| WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
| DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE | ||
| FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL | ||
| DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR | ||
| SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | ||
| CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, | ||
| OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
| OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| ================================================================================ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
133 changes: 133 additions & 0 deletions
133
lib/sdk/server-ai/src/main/java/com/launchdarkly/sdk/server/ai/internal/Interpolator.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| package com.launchdarkly.sdk.server.ai.internal; | ||
|
|
||
| import com.launchdarkly.sdk.LDContext; | ||
| import com.launchdarkly.sdk.server.ai.internal.mustache.Mustache; | ||
| import com.launchdarkly.sdk.server.ai.internal.mustache.Template; | ||
|
|
||
| import java.util.HashMap; | ||
| import java.util.Map; | ||
| import java.util.concurrent.ConcurrentHashMap; | ||
|
|
||
| /** | ||
| * Renders AI Config message and instruction templates using Mustache, following the cross-SDK | ||
| * interpolation policy shared with other SDKs: | ||
| * <ul> | ||
| * <li><b>No HTML escaping.</b> The escape function is the identity, so {@code {{x}}} and | ||
| * {@code {{{x}}}} render identically and values are emitted verbatim.</li> | ||
| * <li><b>Missing and null variables render as the empty string</b> rather than throwing or | ||
| * leaving the placeholder in place.</li> | ||
| * <li><b>The reserved {@code ldctx} variable</b> is derived from the evaluation context and is | ||
| * merged in last, so it always overrides any caller-supplied {@code ldctx}. Context | ||
| * attributes are addressable as {@code {{ldctx.key}}}, {@code {{ldctx.name}}}, and so on.</li> | ||
| * </ul> | ||
| * <p> | ||
| * Compiled templates are cached, keyed by template text. The class is thread-safe: the Mustache | ||
| * compiler is immutable once configured, compiled {@link Template}s are safe for concurrent | ||
| * execution, and the cache is a {@link ConcurrentHashMap}. | ||
| * <p> | ||
| * This class is an internal implementation detail and is not part of the supported API. | ||
| */ | ||
| public final class Interpolator { | ||
| private final Mustache.Compiler compiler; | ||
| private final ConcurrentHashMap<String, Template> templateCache = new ConcurrentHashMap<>(); | ||
|
|
||
| /** | ||
| * Creates an interpolator with the cross-SDK escaping policy. | ||
| */ | ||
| public Interpolator() { | ||
| // defaultValue("") makes both missing variables and variables that resolve to null render as | ||
| // the empty string (it sets jmustache's missingIsNull=true and nullValue=""). escapeHTML(false) | ||
| // emits values verbatim, matching the JS/Python SDKs. | ||
| this.compiler = Mustache.compiler() | ||
| .escapeHTML(false) | ||
| .defaultValue(""); | ||
| } | ||
|
|
||
| /** | ||
| * Renders a template with the given variables and evaluation context. | ||
| * | ||
| * @param template the template text; if {@code null} the result is {@code null} | ||
| * @param variables caller-supplied variables; may be {@code null} | ||
| * @param context the evaluation context, exposed to the template as {@code ldctx}; may be | ||
| * {@code null} | ||
| * @return the rendered string, or {@code null} if {@code template} is {@code null} | ||
| */ | ||
| public String interpolate(String template, Map<String, Object> variables, LDContext context) { | ||
| if (template == null) { | ||
| return null; | ||
| } | ||
| Map<String, Object> merged = new HashMap<>(); | ||
| if (variables != null) { | ||
| merged.putAll(variables); | ||
| } | ||
| // ldctx is added last so it always wins over any caller-supplied "ldctx" entry. | ||
| merged.put("ldctx", contextToMap(context)); | ||
| return render(template, merged); | ||
| } | ||
|
|
||
| /** | ||
| * Renders a template with an already-assembled variable map (no {@code ldctx} injection). | ||
| * | ||
| * @param template the template text; if {@code null} the result is {@code null} | ||
| * @param variables the variables; may be {@code null} | ||
| * @return the rendered string, or {@code null} if {@code template} is {@code null} | ||
| */ | ||
| public String interpolate(String template, Map<String, Object> variables) { | ||
| if (template == null) { | ||
| return null; | ||
| } | ||
| return render(template, variables == null ? new HashMap<String, Object>() : variables); | ||
| } | ||
|
|
||
| private String render(String template, Map<String, Object> variables) { | ||
| Template compiled = templateCache.computeIfAbsent(template, compiler::compile); | ||
| return compiled.execute(variables); | ||
| } | ||
|
|
||
| /** | ||
| * Encodes the evaluation context directly into the nested map structure exposed to templates as | ||
| * {@code ldctx}, without round-tripping through JSON serialization. A single-kind context becomes | ||
| * a map of its attributes; a multi-kind context becomes {@code {"kind":"multi", <kind>: {...}}} | ||
| * with one nested map per individual context. | ||
| */ | ||
| private static Map<String, Object> contextToMap(LDContext context) { | ||
| if (context == null || !context.isValid()) { | ||
| return new HashMap<>(); | ||
| } | ||
| if (context.isMultiple()) { | ||
| Map<String, Object> map = new HashMap<>(); | ||
| map.put("kind", "multi"); | ||
| int count = context.getIndividualContextCount(); | ||
| for (int i = 0; i < count; i++) { | ||
| LDContext individual = context.getIndividualContext(i); | ||
| if (individual != null) { | ||
| // Mirror LaunchDarkly's standard context JSON: the per-kind objects nested under a | ||
| // multi-kind context omit "kind" because it is already implied by the property key. | ||
| map.put(individual.getKind().toString(), singleContextToMap(individual, false)); | ||
| } | ||
|
cursor[bot] marked this conversation as resolved.
|
||
| } | ||
| return map; | ||
| } | ||
| return singleContextToMap(context, true); | ||
| } | ||
|
|
||
| private static Map<String, Object> singleContextToMap(LDContext context, boolean includeKind) { | ||
| Map<String, Object> map = new HashMap<>(); | ||
| if (includeKind) { | ||
| map.put("kind", context.getKind().toString()); | ||
| } | ||
| map.put("key", context.getKey()); | ||
| if (context.getName() != null) { | ||
| map.put("name", context.getName()); | ||
| } | ||
| if (context.isAnonymous()) { | ||
| map.put("anonymous", true); | ||
| } | ||
| // Custom attribute values can be arbitrary JSON; convert each LDValue to a plain Java value | ||
| // (depth-capped) so nested objects/arrays remain addressable from templates. | ||
| for (String attribute : context.getCustomAttributeNames()) { | ||
| map.put(attribute, LDValueConverter.toJavaObject(context.getValue(attribute))); | ||
| } | ||
| return map; | ||
| } | ||
| } | ||
180 changes: 180 additions & 0 deletions
180
...ver-ai/src/main/java/com/launchdarkly/sdk/server/ai/internal/mustache/BasicCollector.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,180 @@ | ||
| // Vendored from com.samskivert:jmustache:1.15 (BSD 3-Clause, Copyright (c) 2010 Michael Bayne). | ||
| // Relocated to com.launchdarkly.sdk.server.ai.internal.mustache for supply-chain hardening (AIC-2695). | ||
| // Upstream: https://github.com/samskivert/jmustache -- unmodified except for this banner and the package | ||
| // declaration below. See THIRD-PARTY-NOTICES.txt for the full license text. | ||
| // | ||
| // | ||
| // JMustache - A Java implementation of the Mustache templating language | ||
| // http://github.com/samskivert/jmustache/blob/master/LICENSE | ||
|
|
||
| package com.launchdarkly.sdk.server.ai.internal.mustache; | ||
|
|
||
| import java.util.Collections; | ||
| import java.util.Iterator; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
| import java.util.NoSuchElementException; | ||
|
|
||
| /** | ||
| * A collector that does not use reflection and can be used with GWT. | ||
| */ | ||
| public abstract class BasicCollector implements Mustache.Collector | ||
| { | ||
| public Iterator<?> toIterator (final Object value) { | ||
| if (value instanceof Iterable<?>) { | ||
| return ((Iterable<?>)value).iterator(); | ||
| } | ||
| if (value instanceof Iterator<?>) { | ||
| return (Iterator<?>)value; | ||
| } | ||
| if (value.getClass().isArray()) { | ||
| final ArrayHelper helper = arrayHelper(value); | ||
| return new Iterator<Object>() { | ||
| private int _count = helper.length(value), _idx; | ||
| @Override public boolean hasNext () { return _idx < _count; } | ||
| @Override public Object next () { return helper.get(value, _idx++); } | ||
| @Override public void remove () { throw new UnsupportedOperationException(); } | ||
| }; | ||
| } | ||
| return null; | ||
| } | ||
|
|
||
| public Mustache.VariableFetcher createFetcher (Object ctx, String name) { | ||
| if (ctx instanceof Mustache.CustomContext) return CUSTOM_FETCHER; | ||
| if (ctx instanceof Map<?,?>) return MAP_FETCHER; | ||
|
|
||
| // if the name looks like a number, potentially use one of our 'indexing' fetchers | ||
| char c = name.charAt(0); | ||
| if (c >= '0' && c <= '9') { | ||
| if (ctx instanceof List<?>) return LIST_FETCHER; | ||
| if (ctx instanceof Iterator<?>) return ITER_FETCHER; | ||
| if (ctx.getClass().isArray()) return arrayHelper(ctx); | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| /** This should return a thread-safe map, either {@link Collections#synchronizedMap} called on | ||
| * a standard {@link Map} implementation or something like {@code ConcurrentHashMap}. */ | ||
| public abstract <K,V> Map<K,V> createFetcherCache (); | ||
|
|
||
| protected static ArrayHelper arrayHelper (Object ctx) { | ||
| if (ctx instanceof Object[]) return OBJECT_ARRAY_HELPER; | ||
| if (ctx instanceof boolean[]) return BOOLEAN_ARRAY_HELPER; | ||
| if (ctx instanceof byte[]) return BYTE_ARRAY_HELPER; | ||
| if (ctx instanceof char[]) return CHAR_ARRAY_HELPER; | ||
| if (ctx instanceof short[]) return SHORT_ARRAY_HELPER; | ||
| if (ctx instanceof int[]) return INT_ARRAY_HELPER; | ||
| if (ctx instanceof long[]) return LONG_ARRAY_HELPER; | ||
| if (ctx instanceof float[]) return FLOAT_ARRAY_HELPER; | ||
| if (ctx instanceof double[]) return DOUBLE_ARRAY_HELPER; | ||
| return null; | ||
| } | ||
|
|
||
| protected static final Mustache.VariableFetcher CUSTOM_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| Mustache.CustomContext custom = (Mustache.CustomContext)ctx; | ||
| Object val = custom.get(name); | ||
| return val == null ? Template.NO_FETCHER_FOUND : val; | ||
| } | ||
| @Override public String toString () { | ||
| return "CUSTOM_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static final Mustache.VariableFetcher MAP_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| Map<?,?> map = (Map<?,?>)ctx; | ||
| if (map.containsKey(name)) return map.get(name); | ||
| // special case to allow map entry set to be iterated over | ||
| if ("entrySet".equals(name)) return map.entrySet(); | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| @Override public String toString () { | ||
| return "MAP_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static final Mustache.VariableFetcher LIST_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| try { | ||
| return ((List<?>)ctx).get(Integer.parseInt(name)); | ||
| } catch (NumberFormatException nfe) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } catch (IndexOutOfBoundsException e) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| } | ||
| @Override public String toString () { | ||
| return "LIST_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static final Mustache.VariableFetcher ITER_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| try { | ||
| Iterator<?> iter = (Iterator<?>)ctx; | ||
| for (int ii = 0, ll = Integer.parseInt(name); ii < ll; ii++) iter.next(); | ||
| return iter.next(); | ||
| } catch (NumberFormatException nfe) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } catch (NoSuchElementException e) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| } | ||
| @Override public String toString () { | ||
| return "ITER_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static abstract class ArrayHelper implements Mustache.VariableFetcher { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| try { | ||
| return get(ctx, Integer.parseInt(name)); | ||
| } catch (NumberFormatException nfe) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } catch (ArrayIndexOutOfBoundsException e) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| } | ||
| public abstract int length (Object ctx); | ||
| protected abstract Object get (Object ctx, int index); | ||
| } | ||
|
|
||
| protected static final ArrayHelper OBJECT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((Object[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((Object[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper BOOLEAN_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((boolean[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((boolean[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper BYTE_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((byte[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((byte[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper CHAR_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((char[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((char[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper SHORT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((short[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((short[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper INT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((int[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((int[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper LONG_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((long[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((long[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper FLOAT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((float[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((float[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper DOUBLE_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((double[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((double[])ctx).length; } | ||
| }; | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was only added to give attribution to the vendored lib but if theres existing attribution process, I'm happy to switch to that.