Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added 'long' as a target type for 'convert_entry_type' processor #4359

Merged
merged 8 commits into from
Apr 4, 2024
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

package org.opensearch.dataprepper.typeconverter;

public class LongConverter implements TypeConverter<Long> {
public Long convert(Object source) throws IllegalArgumentException {
if (source instanceof String) {
return Long.parseLong((String)source);
}
if (source instanceof Float) {
return (long)(float)((Float)source);
}
if (source instanceof Double) {
return (long)(double)((Double)source);
}
if (source instanceof Boolean) {
return ((Boolean)source) ? 1L : 0L;
}
if (source instanceof Integer) {
return ((Integer)source).longValue();
}
if (source instanceof Long) {
return (Long)source;
}
throw new IllegalArgumentException("Unsupported type conversion. Source class: " + source.getClass());
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

package org.opensearch.dataprepper.typeconverter;

import org.junit.jupiter.api.Test;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;

public class LongConverterTests {
@Test
void testStringToLongConversion() {
LongConverter converter = new LongConverter();
final String stringConstant = "1234";
assertThat(converter.convert(stringConstant), equalTo(Long.parseLong(stringConstant)));
}
@Test
void testfloatToLongConversion() {
LongConverter converter = new LongConverter();
final Float floatConstant = (float)1234.56789;
assertThat(converter.convert(floatConstant), equalTo((long)(float)floatConstant));
}
@Test
void testDoubleToLongConversion() {
LongConverter converter = new LongConverter();
final Double doubleConstant = (double)12345678.12345678;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be good to have a test that converts a value which is larger than the maximum integer in addition to this one. You could make a large double constant like 2.0 * Integer.MAX_INT.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test case testDoubleToLongConversionWithBigValue for it.

assertThat(converter.convert(doubleConstant), equalTo((long)(double)doubleConstant));
}
@Test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These test cases look good. One thing you could do to make it easier to add more tests would be to use @ParameterizedTest with an Arguments source. An example of that can be found here (

void testArithmeticExpressionEvaluatorInvalidInput(final String expression, final Event event) {
)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the ParameterizedTest for all the test except for testInvalidStringConversion as had to create a MethodSource.

void testBooleanToLongConversion() {
LongConverter converter = new LongConverter();
final Boolean boolFalseConstant = false;
assertThat(converter.convert(boolFalseConstant), equalTo(0L));
final Boolean boolTrueConstant = true;
assertThat(converter.convert(boolTrueConstant), equalTo(1L));
}
@Test
void testLongToLongConversion() {
LongConverter converter = new LongConverter();
final Long longConstant = (long)1234;
assertThat(converter.convert(longConstant), equalTo(longConstant));
}
@Test
void testInvalidStringConversion() {
IntegerConverter converter = new IntegerConverter();
assertThrows(IllegalArgumentException.class, () -> converter.convert(new Object()));
IntegerConverter converter = new IntegerConverter();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing the following failure:

/home/runner/work/data-prepper/data-prepper/data-prepper-api/src/test/java/org/opensearch/dataprepper/typeconverter/LongConverterTests.java:51: error: variable converter is already defined in method testInvalidStringConversion()
         IntegerConverter converter = new IntegerConverter();

I think you can remove lines 51 and 52 and they repeat the same tests from 49-50.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was a mistake on my part of using IntegerConverter in LongConverterTests. Fixed it.

assertThrows(IllegalArgumentException.class, () -> converter.convert(new Object()));
}
}

2 changes: 1 addition & 1 deletion data-prepper-plugins/mutate-event-processors/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ and the type conversion processor will change it to the following output, where
### Configuration
* `key` - keys whose value needs to be converted to a different type. Required if `keys` option is not defined.
* `keys` - list of keys whose value needs to be converted to a different type. Required if `key` option is not defined.
* `type` - target type for the value of the key. Possible values are `integer`, `double`, `string`, and `boolean`. Default is `integer`.
* `type` - target type for the value of the key. Possible values are `integer`, `double`, `long`, `string`, and `boolean`. Default is `integer`.
* `null_values` - treat any value in the null_values list as null.
* Example: `null_values` is `["-"]` and `key` is `key1`. `{"key1": "-", "key2": "value2"}` will parse into `{"key2": "value2"}`
* `tags_on_failure`(Optional)- A `List` of `String`s that specifies the tags to be set in the event the processor fails to convert `key` or `keys` to configured `type`. These tags may be used in conditional expressions in other parts of the configuration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import org.opensearch.dataprepper.typeconverter.StringConverter;
import org.opensearch.dataprepper.typeconverter.DoubleConverter;
import org.opensearch.dataprepper.typeconverter.BooleanConverter;
import org.opensearch.dataprepper.typeconverter.LongConverter;

import java.util.Map;
import java.util.Arrays;
Expand All @@ -20,7 +21,8 @@ public enum TargetType {
INTEGER("integer", new IntegerConverter()),
STRING("string", new StringConverter()),
DOUBLE("double", new DoubleConverter()),
BOOLEAN("boolean", new BooleanConverter());
BOOLEAN("boolean", new BooleanConverter()),
LONG("long", new LongConverter());

private static final Map<String, TargetType> OPTIONS_MAP = Arrays.stream(TargetType.values())
.collect(Collectors.toMap(
Expand Down
Loading