Why is this necessary?
Convenient delivery of content to the end user, bypassing the hassle with FTP and the inability (due to NDA) to use public services and clouds for file transfer (BTsync, Google- \ Mail- \ Yandex-Disk \ Dropbox \ etc).
Foreword
Our office has a certain infrastructure, which includes Active Directory, in which we have employees who are in groups.
During a session on a PC, after login, by means of configured policies, a network drive is mounted for each employee. Through it, from time to time, data is exchanged. The disk has a specific folder structure, the rights to which are configured through these same groups. Everyone sees what the settings allow him.
Usually, to send data to the outside world, either a separate FTP is raised, or the data is uploaded to an existing one. I admit, this is far from always convenient - at least, creating and managing temporary logins and passwords (for example, when partners need to give ready content), controlling space on the FTP server, manually uploading to an FTP server before "sending".
At some point, we needed to be able to directly give end users data to the outside world from our network drive and, preferably, it should be relatively safe: you can create a download link (it had a lifetime, a password, differentiation of rights to the original data and so on).
This idea was pushed to us by one of the partners, since they had an internal service, but there it was self-written.
There were no ready-made options on the fly in the search engine (if you have them in mind, please unsubscribe in the comments), and no one had the desire and resources to spend potentially many man-hours on development from scratch, testing and support. And why reinvent the wheel, if often everything has already been invented before. This is how the NextCloud service came to mind, which knows how to connect external resources to itself.
We will talk about the latest stable, currently version 19, but our setup method will work for earlier versions as well - we initially implemented it on version 16 and then gradually updated. Recently I just raised it from scratch on the last (19), and based on it I am writing an article.
What we want to get in the end:
- ActiveDirectoty\LDAP, , Jira, Confluence, Nexus .
- NextCloud, , , .
- NextCloud — - , . , .
- -, , NextCloud .* , — .
- An employee can create temporary password-protected links to any resources available to him - be it a folder or a separate file. And also manage them (links) - revoke, change the lifespan, etc.
- The end user, to whom the secure link was sent, only needs to open it and enter the password to be able to download the data shared with him.
Expanding and configuring dependencies
To begin with, we need to have a separate virtual machine or server where we can install the operating system, and then NextCloud.
There is more than one article on Habré devoted to the deployment of the system and service.
AlexanderSquite well and described in detail the process from the installation of the system to the very configuration of the cloud (including updating the articles by year). I see no reason to repeat all this again.
1. Since we are connecting a network drive to NextCloud, we need packages to the system: smbclient , libsmbclient , php-ldap , and php-smbclient .
Remark, in case of using Docker
2. Due to the peculiarities of the settings of our samba server (smb1 support is disabled), on the machine with nextcloud, in the files /etc/samba/smb.conf and /usr/share/samba/smb.conf, we had to change the lines responsible for the protocol:
[global]
client min protocol = SMB2
client max protocol = SMB3
Continuation of the Dockerfile example
RUN rm -frv /etc/samba/smb.conf /usr/share/samba/smb.conf
ADD smb.conf /etc/samba/
ADD smb.conf /usr/share/samba/Otherwise, nextcloud was never able to connect to the drive.
Nextcloud setup
So, Nextcloud is already installed, the dependencies are installed, and the service has one internal user that was created during the installation.
Step one. Preparing a template for employee accounts.
Since we will have more than one employee in the system, and gradually their number will change - if you do not pre-configure the template of the user being created - everyone will have several example files in their home folder. It's good that nextcloud has a separate setting for this - skeleton files , which is configured in config.php .
'skeletondirectory' => '/var/www/html/data/donotdeletme',
That is, you can create an empty folder and specify the full path to it in the config.
Step two. Making users "read-only"
It is enough to specify the quota in "1 B" (1 byte) in the user settings section ( http (s): //nextcloud.domain.tld/settings/users ).
Step three - fix the ZipStreamer in advance
ZipStreamer is a library used in the NextCloud backend, it is used to create archives on the fly, that is, while downloading a bunch of files.
How it works
, "Dowload All Files"/" ", , ( ) , .
, ( .zip, — .tar)
, , .
: Google Drive, ., ..
, ( .zip, — .tar)
, , .
: Google Drive, ., ..
The problem lies in the fact that, for unknown reasons, the logic of switching zip to zip64 embedded in NextCloud fails, and if there are more than 65536 files in the folder, and / or their total weight exceeds 4GB, you will most likely face a problem that the downloaded the file will be either beaten, or the download will be interrupted after downloaded 4GB.
This problem was allocated enough time, on GitHub there is even, and not one, closed ticket (# 1755 , # 15871 , # 8798 ), but despite the fact that the problem was supposedly solved , we still have it, and reproduced with varying success , greatly interfering with work. I had to solve it more radically.
Decision
100% 64 , 64- . « ».
<, nextcloud>/lib/private/Streamer.php:
, -, , .
.
<, nextcloud>/lib/private/Streamer.php:
- $this->streamerInstance = new ZipStreamer(['zip64' => false]);
+ $this->streamerInstance = new ZipStreamer(['zip64' => true]);
, Docker
Integrity Check ( /usr/src/nextcloud/* /var/www/html/*).
/var/www — , .
/usr/src/nextcloud/lib/private/ CI . , .
/var/www — , .
/usr/src/nextcloud/lib/private/ CI . , .
Dockerfile
RUN rm -frv /usr/src/nextcloud/lib/private/Streamer.php
ADD Streamer.php /usr/src/nextcloud/lib/private/
RUN chown nobody:nogroup /usr/src/nextcloud/lib/private/Streamer.php
, -, , .
.
Step four - adjust link parameters
In the publication parameters, we configure the rules as it is convenient. For example, we make mandatory password protection and set a mandatory lifetime.
Step five - connect the network drive
- Let's go to setting up external storages . ( http (s): //nextcloud.domain.tld/settings/admin/externalstorages )
- We choose to add SMB \ CIFS storage.
- We fill in the fields for name, domain, folder, etc.
- We select "Credentials, store in the database" - it is this item that allows the user to connect the disk to his account in NextCloud when the user enters using his login and password link. (The item storing the login and password during the session did not take off).
- Do not forget in the menu <...> to mark the "Read-only" checkboxes and the permission for sharing.
Step six - launching users through LDAP
Now that we have everything ready, we install the plugin from the marketplace, and immediately after we connect LDAP. In our system, we gave access to employees who are in the NextcloudAccess group . You can do the same.
Conclusion
That's it, after all these simple, but sometimes not the most obvious manipulations - the service is working, the disk is connected, employees are added, and users are downloading and happy.
Complete example of our Dockerfile
FROM nextcloud:latest
ENV DEBIAN_FRONTEND noninteractive
#installing smbclient
RUN apt update -y && apt install -y --allow-unauthenticated smbclient libsmbclient libsmbclient-dev
RUN pecl install smbclient
RUN docker-php-ext-enable smbclient
#fix of ZipStreammer
RUN rm -frv /usr/src/nextcloud/lib/private/Streamer.php
ADD Streamer.php /usr/src/nextcloud/lib/private/
RUN chown nobody:nogroup /usr/src/nextcloud/lib/private/Streamer.php
#fix of smb config
RUN rm -frv /etc/samba/smb.conf /usr/share/samba/smb.conf
ADD smb.conf /etc/samba/
ADD smb.conf /usr/share/samba/Stress Testing
"How are you doing with the load?" - you ask at last.
Measurements during rush hour
Our service instance runs on ~ 6Gb Ram + 6CPU in a virtual machine among other VMs.
At the peak of the network load, a little more than 2.5Gb of RAM was used, the processor was not clogged, and the return on average was about 5Gbit / s (the record reached 8Gbit / s).
The only thing that we noticed is that when we transfer over 6Gbit / s to the outside world, our web interface periodically falls off, but the downloads from users continue to go.
Disadvantages identified:
- NextCloud has no global control over all created links. They are displayed only within each account in a separate section.
- If you share the root of a connected drive using a link, this can serve as a potential data leak, since all folders available to your account will be shared in the link.
- I noticed that for a year of using the raised service, at the moment, the oc_filecache table in the database weighs ~ 29Gb and has about ~ 100k rows (we are using vanilla MySQL 5.7.x). This is due to bugs # 16834 , # 6395 , # 7312 , # 20349 . While there is surveillance on the second instance.
Thanks for reading. If you have any questions, comments or suggestions - do not hesitate, I am always glad to constructive criticism.
: nextcloud 16, 17, 18, 19
: 26.07.2020
: 24.09.2020
: . Areso.
: 1.0.0.9