File transfer - Revision history

Haars0011 at 08:23, 28 January 2025

2025-01-28T08:23:08Z

← Older revision		Revision as of 10:23, 28 January 2025
Line 94:		Line 94:
	rclone copy --create-empty-src-dirs --copy-links --progress bin/ remote:test		rclone copy --create-empty-src-dirs --copy-links --progress bin/ remote:test
	</pre>		</pre>

			Another nice tool is rclone ncdu : https://rclone.org/commands/rclone_ncdu/

	As soon as your are done, please remove your config:		As soon as your are done, please remove your config:
	<pre>		<pre>

Haars0011 at 13:09, 14 January 2025

2025-01-14T13:09:13Z

← Older revision		Revision as of 15:09, 14 January 2025
Line 79:		Line 79:
	You will then have to create an new config (I will use remote as name, but you can choose what you want), and select '''''onedrive''''' as the type.		You will then have to create an new config (I will use remote as name, but you can choose what you want), and select '''''onedrive''''' as the type.
	Use defaults for next few steps, and choose '''''global''''' for the region.		Use defaults for next few steps, and choose '''''global''''' for the region.
	Don't start the advanced config, and do use auto config.		Don't start the advanced config, and do use auto config ("Use web browser to automatically authenticate rclone with remote?").
	'''Copy the URL''' to your local web browser, and enter your WUR credentials when asked.		'''Copy the URL''' to your local web browser, and enter your WUR credentials when asked.
	In the next steps, you will have to select onedrive, the WUR uses the Business version.		In the next steps, you will have to select onedrive, the WUR uses the Business version.

Sun092 at 13:41, 4 June 2024

2024-06-04T13:41:01Z

← Older revision		Revision as of 15:41, 4 June 2024
Line 70:		Line 70:
	ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682		ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682
	</pre>		</pre>
	* For Windows PUTTY users: [https://wiki.anunna.wur.nl/index.php?title=Putty_SSH_Tunnelling Putty SSH Tunnelling]		* For Windows PUTTY users, please refer to: [https://wiki.anunna.wur.nl/index.php?title=Putty_SSH_Tunnelling Putty SSH Tunnelling]

	Then load the rclone module (see above), and start the configure process by entering		Then load the rclone module (see above), and start the configure process by entering

Sun092 at 12:37, 4 June 2024

2024-06-04T12:37:29Z

← Older revision		Revision as of 14:37, 4 June 2024
Line 70:		Line 70:
	ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682		ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682
	</pre>		</pre>
	For Windows PUTTY users: https://wiki.anunna.wur.nl/index.php?title=Putty_SSH_Tunnelling~~&action=edit~~		* For Windows PUTTY users: [https://wiki.anunna.wur.nl/index.php?title=Putty_SSH_Tunnelling Putty SSH Tunnelling]

	Then load the rclone module (see above), and start the configure process by entering		Then load the rclone module (see above), and start the configure process by entering

Sun092 at 12:33, 4 June 2024

2024-06-04T12:33:53Z

← Older revision		Revision as of 14:33, 4 June 2024
Line 70:		Line 70:
	ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682		ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682
	</pre>		</pre>
			For Windows PUTTY users: https://wiki.anunna.wur.nl/index.php?title=Putty_SSH_Tunnelling&action=edit

	Then load the rclone module (see above), and start the configure process by entering		Then load the rclone module (see above), and start the configure process by entering

Haars0011 at 13:58, 3 June 2024

2024-06-03T13:58:16Z

← Older revision		Revision as of 15:58, 3 June 2024
Line 64:		Line 64:

	To be able to access your own OneDrive space, you will need to configure rclone.		To be able to access your own OneDrive space, you will need to configure rclone.
	In one of the steps, rclone starts a webserver, so we will have to use SSH to create a tunnel to the login server, so we can access it on your computer.		In one of the steps, rclone starts a webserver, so we will '''first''' have to use SSH to create a tunnel to the login server, so we can access the webserver on your own computer.

	So first connect to anunna, with the tunnel:		So first connect to anunna, with the tunnel:
Line 79:		Line 79:
	Use defaults for next few steps, and choose '''''global''''' for the region.		Use defaults for next few steps, and choose '''''global''''' for the region.
	Don't start the advanced config, and do use auto config.		Don't start the advanced config, and do use auto config.
	Copy the URL to your local web browser, and enter your WUR credentials when asked.		'''Copy the URL''' to your local web browser, and enter your WUR credentials when asked.
	In the next steps, you will have to select onedrive, the WUR uses the Business version.		In the next steps, you will have to select onedrive, the WUR uses the Business version.
	If all is well, you now have a working config.		If all is well, you now have a working config.

Haars0011 at 14:58, 18 April 2024

2024-04-18T14:58:04Z

← Older revision		Revision as of 16:58, 18 April 2024
Line 41:		Line 41:

	== Samba/CIFS based protocols ==		== Samba/CIFS based protocols ==
	The Common Interface File System ([http://en.wikipedia.org/wiki/Cifs CIFS]) is commonly used in and between Windows systems for file sharing. It is only available to clients within WURnet.		The Common Interface File System ([http://en.wikipedia.org/wiki/Cifs CIFS]) is commonly used in and between Windows systems for file sharing. It is only available to clients within WURnet or when connected through the VPN.

	There are two mount points that are available		There are two mount points that are available

Haars0011 at 14:33, 8 February 2024

2024-02-08T14:33:10Z

← Older revision		Revision as of 16:33, 8 February 2024
Line 49:		Line 49:
	You can enter these in the location bar of File Explorer.		You can enter these in the location bar of File Explorer.

			If you have issues with logging in, the issue is probably your username. In that case, there are a couple of options you could try:
			# Use '\WUR\[userid]' as username
			# Use '[userid]' as username
			# Use '[WUR e-mail]' as username

			All these can be entered if you select "connect using different credentials" in the connection box.
	== rclone to OneDrive ==		== rclone to OneDrive ==

Prins0891: to

 to fix code blocks

2023-06-15T14:44:27Z

<source> to <pre> to fix code blocks

← Older revision		Revision as of 16:44, 15 June 2023
Line 6:		Line 6:
	Syntax of the scp command requires from-to order:		Syntax of the scp command requires from-to order:

	<~~source lang='bash'~~>		<pre>
	scp <source> <destination>		scp <source> <destination>
	</~~source~~>		</pre>

	<~~source lang='bash'~~>		<pre>
	scp -pr /home/WUR/[username]/folder_to_transfer [username]@login.anunna.wur.nl:/lustre/scratch/WUR/ABGC/		scp -pr /home/WUR/[username]/folder_to_transfer [username]@login.anunna.wur.nl:/lustre/scratch/WUR/ABGC/
	</~~source~~>		</pre>

	This example assumes a user that is part of the ABGC user group. See the [[Lustre_PFS_layout \| Lustre Parallel File System layout]] page for further details. The -p flag will preserve the file metadata such as timestamps. The -r flag allows for recursive copying. Further options can be found in the [http://en.wikipedia.org/wiki/Man_page man pages].		This example assumes a user that is part of the ABGC user group. See the [[Lustre_PFS_layout \| Lustre Parallel File System layout]] page for further details. The -p flag will preserve the file metadata such as timestamps. The -r flag allows for recursive copying. Further options can be found in the [http://en.wikipedia.org/wiki/Man_page man pages].

	<~~source lang='bash'~~>		<pre>
	man scp		man scp
	</~~source~~>		</pre>

	=== rsync ===		=== rsync ===
	The [http://en.wikipedia.org/wiki/Rsync rsync protocol], like the scp protocol, allow CLI-based copying of files. The rsync protocol, however, will only transfer those files between systems that have changed, i.e. it synchronises the files, hence the name. The rsync protocol is very well suited for making regular backups and file syncs between file systems. Like the scp command, syntax is in the from-to order.		The [http://en.wikipedia.org/wiki/Rsync rsync protocol], like the scp protocol, allow CLI-based copying of files. The rsync protocol, however, will only transfer those files between systems that have changed, i.e. it synchronises the files, hence the name. The rsync protocol is very well suited for making regular backups and file syncs between file systems. Like the scp command, syntax is in the from-to order.
	<~~source lang='bash'~~>		<pre>
	rsync <source> <destination>		rsync <source> <destination>
	</~~source~~>		</pre>
	e.g.:		e.g.:
	<~~source lang='bash'~~>		<pre>
	rsync -av /home/WUR/[username]/folder_to_transfer [username]@login.anunna.wur.nl:/lustre/scratch/WUR/ABGC/		rsync -av /home/WUR/[username]/folder_to_transfer [username]@login.anunna.wur.nl:/lustre/scratch/WUR/ABGC/
	</~~source~~>		</pre>
	The -a flag will preserve file metadata and allows for recursive copying, amongst others. The -v flag provides verbose output. Further options can be found in the [http://en.wikipedia.org/wiki/Man_page man pages].		The -a flag will preserve file metadata and allows for recursive copying, amongst others. The -v flag provides verbose output. Further options can be found in the [http://en.wikipedia.org/wiki/Man_page man pages].
	<~~source lang='bash'~~>		<pre>
	man scp		man scp
	</~~source~~>		</pre>

	=== WinSCP ===		=== WinSCP ===
Line 53:		Line 53:
	To easily transfer data to OneDrive, one can use '''''rclone'''''.		To easily transfer data to OneDrive, one can use '''''rclone'''''.
	It is usable through the modules system, so you can access it through		It is usable through the modules system, so you can access it through
	<~~source lang='bash'~~>		<pre>
	module load rclone		module load rclone
	</~~source~~>		</pre>

	To be able to access your own OneDrive space, you will need to configure rclone.		To be able to access your own OneDrive space, you will need to configure rclone.
Line 61:		Line 61:

	So first connect to anunna, with the tunnel:		So first connect to anunna, with the tunnel:
	<~~source lang='bash'~~>		<pre>
	ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682		ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682
	</~~source~~>		</pre>

	Then load the rclone module (see above), and start the configure process by entering		Then load the rclone module (see above), and start the configure process by entering
	<~~source lang='bash'~~>		<pre>
	rclone config		rclone config
	</~~source~~>		</pre>

	You will then have to create an new config (I will use remote as name, but you can choose what you want), and select '''''onedrive''''' as the type.		You will then have to create an new config (I will use remote as name, but you can choose what you want), and select '''''onedrive''''' as the type.
Line 78:		Line 78:

	To test. do this:		To test. do this:
	<~~source lang='bash'~~>		<pre>
	rclone tree remote:		rclone tree remote:
	</~~source~~>		</pre>

	If all is well, you see the content of your own OneDrive.		If all is well, you see the content of your own OneDrive.
	Now you can create folders, and copy data to and from your OneDrive:		Now you can create folders, and copy data to and from your OneDrive:
	<~~source lang='bash'~~>		<pre>
	rclone copy --create-empty-src-dirs --copy-links --progress bin/ remote:test		rclone copy --create-empty-src-dirs --copy-links --progress bin/ remote:test
	</~~source~~>		</pre>
	As soon as your are done, please remove your config:		As soon as your are done, please remove your config:
	<~~source lang='bash'~~>		<pre>
	rclone config delete remote		rclone config delete remote
	</~~source~~>		</pre>
	Anunna is relatively safe, but better safe than sorry.		Anunna is relatively safe, but better safe than sorry.

Haars001 at 19:10, 10 February 2021

2021-02-10T19:10:32Z

← Older revision		Revision as of 21:10, 10 February 2021
Line 102:		Line 102:
	* [https://filezilla-project.org FileZilla homepage]		* [https://filezilla-project.org FileZilla homepage]
	* [http://en.wikipedia.org/wiki/Cifs The Common Interface File System (CIFS) on Wikipedia]		* [http://en.wikipedia.org/wiki/Cifs The Common Interface File System (CIFS) on Wikipedia]
			* [https://rclone.org/onedrive/ Info on adding Microsoft OneDrive from the rclone website]