8. Can I load several files without merging them beforehand?

9 June 2018June 9, 2018Nargis Tahara

Spark

In Spark, the method textFile() takes an URI for the file (either a local path or a hdfs:// ). You could run this method on a single file or a directory which may contains more than one file by calling :

1
2

val rdfFile = sc.textFile("hdfs://.../yourdirectory/yourfile.nt")
val rdfDirectory = sc.textFile("hdfs://.../yourdirectory/")