An FS implementation for working with S3

Posted on October 29, 2024 by Mandla Mbuli

I want to read some files from S3 using Scala. I could have just used a HashMap and probably had an easy time and finished quickly. This is not for work though so I took the scenic route

Filename and Directory name

I don’t think with I will ever use this on Windows, so I optimised for Linux file names. I found out, there a limits to File names and Directory names in Linux.

The following commands give the max length for these things.

  getconf NAME_MAX / # 255
  getconf PATh_MAX / # 4096

The challenge here is that UTF-8 does not match 1 to 1 for a characters. So in Scala, I need to create the name and check that the length is not longer that the max.

  // for directory name
  if name.getBytes(StandardCharsets.UTF_8).length > 4096 then Some("dirname is too long") 

  // for file name
  if name.getBytes(StandardCharsets.UTF_8).length > 255 then Some("filename is too long")

In Linux, . as a name is reserved for the current directory and .. is reserved for the parent directory. I don’t know if I will need this in future, so I will just not allow these for both file name and directory name.

    else if name.equals(".") then Some("name cannot be '.'")
    else if name.equals("..") then Some("name cannot be '..'")

The complete code becomes:

type Name = String
type ErrorMessage = String
extension (name: Name)
  def isEmpty: Boolean = name.isEmpty
  def length: Int = name.length
  def isValidDirname: Option[ErrorMessage] =
    if name.isEmpty then Some("dirname cannot be empty")
    else if name.getBytes(StandardCharsets.UTF_8).length > 4096 then Some("dirname is too long")
    else if name.equals(".") then Some("dirname cannot be '.'")
    else if name.equals("..") then Some("dirname cannot be '..'")
    else None
  def isDirname: Boolean = isValidDirname.isEmpty
  def isValidFilename: Option[ErrorMessage] =
    if name.isEmpty then Some("filename cannot be empty")
    else if name.endsWith("/") then Some("filename must not end with a slash")
    else if name.getBytes(StandardCharsets.UTF_8).length > 255 then Some("filename is too long")
    else if name.equals(".") then Some("filename cannot be '.'")
    else if name.equals("..") then Some("filename cannot be '..'")
    else None
  def isFilename: Boolean = isValidFilename.isEmpty

No all Strings can be checked. Which I think makes this a little easier to use.

Dir and File

Next I need