Option to choose output folder of dockerExtraFiles

[mojimi]

Thanks @jdufresne for adding the new feature that adds and documents a way of adding extra base libraries to the lambda.

But at the moment it won't work properly for layers, as the lambda environment will look for so libs in the lib folder, just like it look for python libs in the python folder as you can see here https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path

I don't know much about PR, but judging from the line here : https://github.com/UnitedIncome/serverless-python-requirements/blob/529b9da27930ff57dc061069ab0007d5dc7fe55a/lib/pip.js#L257 , this could be a simple solution (rough idea) :

for (let path of options.dockerExtraFiles) {
    if (typeof path === "string") {
        pipCmds.push(['cp', path, '/var/task/']);
    }else{
        pipCmds.push(['cp'], path.from, '/var/task' + path.to])
    }
}

Which would use the following syntax in yaml :

dockerExtraFiles:
      - /usr/lib64/libcairo-1.0.so.0
      - from : /usr/lib64/libpango-1.0.so.0
         to : /lib/

Problem is, I don't know if this would be enough for a layer, since it might just put into /python/lib

[dschep]

Problem is, I don't know if this would be enough for a layer, since it might just put into /python/lib

Yeah, I think you're right about that: https://github.com/UnitedIncome/serverless-python-requirements/blob/6b5e0e25fcf022e56a2ee15505794e05128efa1e/lib/layer.js#L16

[mojimi]

Yep it definitely needs extra work in the layer.js

I've sketched out a basic idea here (unfinished/untested) :

function zipRequirements() {
  const rootZip = new JSZip();
  const src = path.join('.serverless', 'requirements');
  const runtimepath = 'python';
  const extraPaths = [];

  //Finding dockerExtraFiles with the "to" parameter if using layer
  //Not sure how to bring the options object to here
  if(options.layer && options.dockerExtraFiles){
      for (let path of options.dockerExtraFiles){
          if(path.to){
              extraPaths.push(path);
          }
      }
  }
  //If found any
  if(extraPaths.length > 0){
        extraPathsPromises = []
        for (let extraPath of extraPaths){
            const absExtraPath = path.join(src, extraPath)
            extraPathsPromises.push(
                addTree(rootZip.folder(extraPath), absExtraPath)) //Adding it to the root of the zip
                .then( () => fs.unlink(absExtraPath)) //Removing it from the src path so it doesnt go duplicated into /python/, maybe there's a better way to do this
            )
        }
        return Promise.all(extraPathsPromises)
        .then( () => addTree(rootZip.folder(runtimepath), src))
        .then(() => writeZip(rootZip, path.join('.serverless', 'pythonRequirements.zip')));
  }else{
    return addTree(rootZip.folder(runtimepath), src)
    .then(() => writeZip(rootZip, path.join('.serverless', 'pythonRequirements.zip')));
  }
}

[ptrhck]

Where are the dockerExtraFiles located in the live environment? I have used a Lambda layer to provide .so files which are available in /opt. E.g., in my django settings I do GDAL_LIBRARY_PATH = '/opt/lib/libgdal.so'. What is the equivalent if I use dockerExtraFiles ?

Also, can you wildcard the file names like /opt/lib/*.so?

[razumeiko]

Hi guys. Any plans to push this? CUrrently, I am blocked with this issue