How to use ft_elementwise_product

Question

I am trying to use function ft_elementwise_product which is supposed to make elementwise product, but I can´t, here is a link explaining how the function should work:

http://rstudio.github.io/sparklyr/reference/ft_elementwise_product.html

Having this example:

library(sparklyr)
library(dplyr)
sc <- spark_connect(master = "local")

iris2 = copy_to(sc, iris, "iris2", overwrite = TRUE)

I wanted to multiply Sepal_Length with Sepal_Width, so I tried:

ft_elementwise_product(iris2, Sepal_Length, Sepal_Width, Sepal_Width) 
# object 'Sepal_Length' not found

ft_elementwise_product(iris2, "Sepal_Length", Sepal_Width, Sepal_Width) 
# object 'Sepal_Width' not found

ft_elementwise_product(iris2, "Sepal_Length", "Sepal_Width", Sepal_Width) 
# Error in object %||% default : object 'Sepal_Width' not found

ft_elementwise_product(iris2, "Sepal_Length", "Sepal_Width", "Sepal_Width")
# Error: java.lang.Exception: No matched method found for class org.apache.spark.ml.feature.ElementwiseProduct.setScalingVec

ft_elementwise_product(iris2, "Sepal_Length", "Sepal_Width", "Sepal_Length")
# Error: java.lang.Exception: No matched method found for class org.apache.spark.ml.feature.ElementwiseProduct.setScalingVec

ft_elementwise_product(iris2, "Sepal_Length", "result", "Sepal_Length")
# Error: java.lang.Exception: No matched method found for class org.apache.spark.ml.feature.ElementwiseProduct.setScalingVec

iris2 = mutate(iris2, result = 0)
ft_elementwise_product(iris2, "Sepal_Length", "result", "Sepal_Length")
# Error: java.lang.Exception: No matched method found for class org.apache.spark.ml.feature.ElementwiseProduct.setScalingVec

user_1560062 · Answer

I wanted to multiply Sepal_Length with Sepal_Width

In that case you have not use for ElementwiseProduct. Just use standart multiplication:

iris2 %>% mutate(product = Sepal_Length * Sepal_Width)

How to use ft_elementwise_product

ElementWise product is a ML Transformer used with Pipelines. It is used to multiply Vector columns by local scaling Vectors. With current master (0.7.0-9105) it can be used as shown below:

iris2 %>%  
  # Create Vector column which can be used as an input
  ft_vector_assembler(input.col=colnames(.)[-5], output.col="features") %>% 
  # Transform assembled vector with vector [-1, 1, -1, 1]
  ft_elementwise_product(
    input.col="features", output.col="scaled_features",
    scaling_vec=c(-1, 1, -1, 1))

How to use ft_elementwise_product

1 Answers