Machine Learning – SANSA-Stack

3. How can I use SANSA for Knowledge Graph Embedding Models?

Currently there are two Knowledge Graph Embedding (KGE) models are implemented: TransE [1] and DistMult (Bilinear-Diag) [2].
The following code snippets show you how you can load your dataset and apply cross validation techniques supported on SANSA KGE.

Spark

// dataset to be loaded

val input = "fb15k.txt"

// technique used to split the data

val technique = "holdout"

val k = 5

val data = new Triples(input, "\t", false, false, spark)

// converting the original data to indexData

val indexedData = new ByIndex(data.triples, spark)

val numericData = indexedData.numeric()

val (train, test) = technique match {

case "holdout" => new Holdout(numericData, 0.6f).crossValidation()

case "bootstrapping" => new Bootstrapping(numericData).crossValidation()

case "kFold" => new kFold(numericData, k, spark).crossValidation()

case _ =>

throw new RuntimeException("'" + technique + "' - Not supported, yet.")

}

2. How can I use SANSA for mining rules?

Rule mining for knowledge bases is used to look for new facts, or such rules can be used to identify errors in the knowledge bases. These rules can be used for reasoning, and the rules that define regularities in the data can be used to understand the data better.

SANSA uses AMIE+ algorithm to mine association rules or correlations in the RDF dataset. These rules have the form $r(x,y) <= B1 & B2 & ... Bn$ while r(x,y) is the head and B the body, a conjunction of atoms of the rule. The process starts with rules with only one atom, which are then refined to add more atoms with fresh or previously used variables. The rules is accepted as output rule, if its support is above a support threshold.

Spark

Within spark, the support of a rule is calculated using DataFrames. For the rules with two atoms, the predicates of head and body are filtered against a dataframe, which contains all the instantiated atoms with a particular predicate. Different dataframes are then joined and only the rows with correct variables are kept. For greater sizes, new atoms are joined with the previous dataframes (previously refined rules), which are stored in the parquet format with rules as name for corresponding folders.

import net.sansa_stack.ml.spark.mining.amieSpark.KBObject.KB

import net.sansa_stack.ml.spark.mining.amieSpark.{ RDFGraphLoader, DfLoader }

import net.sansa_stack.ml.spark.mining.amieSpark.MineRules.Algorithm

val know = new KB()

know.sethdfsPath(hdfsPath)

know.setKbSrc(input)

know.setKbGraph(RDFGraphLoader.loadFromFile(know.getKbSrc(), spark.sparkContext, 2))

know.setDFTable(DfLoader.loadFromFileDF(know.getKbSrc, spark.sparkContext, sparkSession.sqlContext, 2))

val algo = new Algorithm(know, 0.01, 3, 0.1, hdfsPath)

//var erg = algo.ruleMining(spark.sparkContext, spark.sqlContext)

//println(erg)

var output = algo.ruleMining(spark.sparkContext, spark.sqlContext)

var outString = output.map { x =>

var rdfTrp = x.getRule()

var temp = ""

for (i <- 0 to rdfTrp.length - 1) {

if (i == 0) {

temp = rdfTrp(i) + " <= "

} else {

temp += rdfTrp(i) + " \u2227 "

}

temp = temp.stripSuffix(" \u2227 ")

temp

}.toSeq

var rddOut = spark.sparkContext.parallelize(outString)

rddOut.saveAsTextFile(outputPath + "/testOut")

Full example code: https://github.com/SANSA-Stack/SANSA-Examples/blob/master/sansa-examples-spark/src/main/scala/net/sansa_stack/examples/spark/ml/mining/MineRules.scala

1. How can I use SANSA for clustering on RDF graph?

SANSA contains the implementation of a partitioning algorithm for RDF graphs given as NTriples. The algorithm uses the structure of the underlying undirected graph to partition the nodes into different clusters. SANSA’s clustering procedure follows a standard algorithm for partitioning undirected graphs aimed to maximize a modularity function, which was first introduced by Newman.

You will need your RDF graph in the form of a text file, with each line containing exactly one triple of the graph. Then you specify the number of iterations and supply a file path where you want your resulting clusters to be saved to.

Spark
Flink

import net.sansa_stack.ml.spark.clustering.RDFByModularityClustering

val numIterations = 100

val input ="path_to_your_RDFgraph.txt"

val output ="path_name_for_clusters.txt"

RDFByModularityClustering(spark.sparkContext, numIterations, input, output)

Full example code: https://github.com/SANSA-Stack/SANSA-Examples/blob/master/sansa-examples-spark/src/main/scala/net/sansa_stack/examples/spark/ml/clustering/RDFByModularityClustering.scala

import net.sansa_stack.ml.flink.clustering.RDFByModularityClustering

val numIterations = 100

val input ="path_to_your_RDFgraph.txt"

val output ="path_name_for_clusters.txt"

val env = ExecutionEnvironment.getExecutionEnvironment

RDFByModularityClustering(env, numIterations, graphFile, outputFile)

Full example code: https://github.com/SANSA-Stack/SANSA-Examples/blob/master/sansa-examples-flink/src/main/scala/net/sansa_stack/examples/flink/ml/clustering/RDFByModularityClustering.scala