GSoC 2016: Weekly Report 1

Hi all,

What did I get done this week?

My initial plan was to work on a concept for implementing new storage types but before the coding period started I had already started trying FTP storage so I continued it for this week as well. I started off with simply adding the credentials in the configuration file and created a new FTPStorage class in storage.py which handles the FTP uploads. I also created a test FTP server on DigitalOcean to test things.

class FTPStorage(StorageAbstract):
 def __init__(self):
 """
 """
 self.ftp_host = config.get_config_value('remote-storage', 'ftp_host')
 self.ftp_user = config.get_config_value('remote-storage', 'ftp_user')
 self.ftp_password = config.get_config_value('remote-storage', 'ftp_password')
 self.target = config.get_config_value('server', 'outputpath')
 self.output_url = '%s%s' % (
 config.get_config_value('server', 'url'),
 config.get_config_value('server', 'outputurl')
 )

 def store(self, output):
 import shutil, tempfile

 file_name = output.file


 (prefix, suffix) = os.path.splitext(file_name)
 if not suffix:
 suffix = output.output_format.get_extension()
 (file_dir, file_name) = os.path.split(prefix)
 output_name = tempfile.mkstemp(suffix=suffix, prefix=file_name,
 dir=self.target)[1]

 full_output_name = os.path.join(self.target, output_name)
 shutil.copy2(output.file, full_output_name)

 ftp = ftplib.FTP(self.ftp_host)
 ftp.login(self.ftp_user, self.ftp_password)
 ftp.storlines("STOR " + output.file, open(full_output_name))
 
 just_file_name = os.path.basename(output_name)

 url = urljoin(self.ftp_host, just_file_name)
 LOGGER.info('File Saved in FTP Server: %s', url)

 return (STORE_TYPE.FTP, output_name, url)

 

Any issues which I faced in this week?

If you look at the above (incomplete) code you can see that I’m still creating a file in the local disk which I upload to the FTP server, because I still haven’t been able to find a solution to create files directly on the remote server. I found some solutions online but I got some weird permission errors when I tried them. It is possible that there are some issues with the FTP server configuration which is causing this error.

What am I planning for the next week?

First of all I will talk to my mentors and decide on how to proceed with giving the user options for the new storage type. I had allocated 2 weeks for implementing FTP storage and since I have already tried FTP storage I think I will have enough time to create a framework for providing new storage options and also to sort out the issue of directly creating the output file on the remote ftp server.

Thank you!