mintCast 318.5 – Syncing Files (mp3)

Download

In our Innards section, we cover backups like Rsync, Nextcloud and some underlying storage.

And finally, community feedback!

INNARDS:

• Joe
  • Rsync
    • Used as the back end for most backup systems 
    • works on most operating systems
    • It has been around for 23 years!
    • Comes stock on nearly every linux operating system
    • I have used it in production on several systems for back ups or syncing data on request
    • fairly standard setup for the command 
      • rsync [options] from to
    • what makes it great is the way that it will remember where you left off with a broken or partial sync.
      • so if you lose your connection you can run the same command again and pick up where you left off
        • especially if you use -P which will allow you to watch the progress and continue from where you left off even if it is mid file
      • The -a flag is great as it preserves symbolic links so you will not get stuck in a loop the way that it was possible to do with scp
    • Plus for backups it will only move files that it can verify have changed which makes it faster and use less data.
      • It will check file sizes and time stamps to see if it needs to sync a new file
      • You can set it to instead check hashes for differences which is much slower but it will find all the changes
      • –checksum is the option to add for this
    • Great to mix with cron for automating backups on your own
      • timing can be a small issue if your backup runs long so don’t set it up to run every minute
  • Resilio sync/btsync/bittorrent sync
    • Does not use rsync
    • uses the bittorrent protocol
    • Started out as bittorrent sync but everyone called it btsync and then when they tried to switch to a paid model they changed to resilio sync
    • The setup is very similar to other syncing programs.  Install the software and setup the user permissions and the places that it will have access to
      • After setup you pick the folders that you want to share
      • then you choose to share it and either a link a code or a qr code is generated
      • you add this to resilio sync on another machine or multiple machines
      • you can choose to limit this by making a single use code or a code that expires after a few days or by leaving it open
      • you can also choose if the other clients will have write permissions on the data or just read
      • you can limit it further on systems by controlling the user permissions and or setting up resilio under its own user
    • Works across multiple OS’s including android
    • The more nodes that you have setup with the data that you are syncing the faster it will be synced across devices.
    • it is self hosted so your data will only be on your computers
    • as for the bittorrent protocol here is how it works
      • Files within the set shared space are broken into small parts then shared then reassembled on the “client side” when all portions are received
      • the smaller files are easier to send then larger files would be
      • so you could be downloading multiple small portions from separate locations at the same time which greatly increases speed based on how many nodes that you have
      • Much like with torrents it is possible for none of the nodes to have a complete copy and still to get a complete copy on all the systems by the end as long as all portions are distributed.
    • it is possible to backup an entire system with resilio sync if you give the group or user enough permissions or run it as root.  
    • better for files that you want to backup but there are pitfalls
      • if you have write permissions on all the nodes then if one of them gets corrupt it will copy across all the systems and all will be corrupt
    • Great to run commands on your home system
      • very tinkery tool when used this way
      • just as an example, set up a cron script to execute a bash script every minute, this bash script looks for a file within the btsync location.  If it is there then you run the commands placed within this file and rename the file with a date stamp so that it is not picked up again(you can also get creative with the output if you want)
      • it is seriously a one line command ‘bash -i /path/to/filename >> filename2’ to execute commands and record output and then mv the filename to doneFilename then echo filename2 >> doneFilename and rm filename2
• Leo
  • The Nextcloud setup was too easy! (For someone that tinkers all the time)
    • sudo snap install nextcloud
    • sudo nextcloud.manual-install leo password
    • sudo nextcloud.occ config:system:set trusted_domains 1 –value=domainname
    • sudo nextcloud.enable-https self-signed
  • If you’re storing your files on external storage like myself, you’ll need to install a removable media plugin.
    • sudo snap connect nextcloud:removable-media
  • The config file lives at
    • sudo nano /var/snap/nextcloud/current/nextcloud/config/config.php
  • Finally, move your data to the new place
    • sudo mv /var/snap/nextcloud/common/nextcloud/data /media/nextcloud/data
  • Nextcloud auto backups
  • In the settings, there is an Auto Upload Photos/Videos toggle.
  • Underneath, define where you want the folder to live, then turn the toggle on. Like magic, Nextcloud starts to suck up all of the files you specify!
  • I haven’t tried the feature with anything other than Photos and Videos, but for a phone, those are typically the types of files you’re trying to backup.
  • For the desktop, the sync is similar. For every device I’ve installed the Nextcloud client on, I have the Desktop directory synced across all the devices. This is fantastic for me since that’s where all my working files go. I just have to ensure that the client is on to do the sync and my files are there, updated, and ready to be edited.
  • Thoughts
  • Overall, the setup is easy if you’ve played with stuff like this before. But, compared to a traditional setup with a manual install of a database, php and webserver, it was extremely easy.
  • The best part of the entire setup is that I’m bad at remembering to do manually backup files with a drag and drop. So, as long as they’re in the right place, I don’t have to do anything!

 

• Tony W
• DISCLAIMER before beginning: This is from a beginner’s perspective.

• • RAID, ZFS Z pools
    • Reasons to use:
      • Allow data redundancy across multiple drives/partitions
      • Can simply mirror a drive
      • Windows has a version of this
      • Use multiple drives/partitions as one volume
    • Typically used for server but can be used for home machines
    • Still recommend backup to other machine or drive
  • RAID
    • RAID can be hardware or software RAID 
      • Hardware RAID is a bit legacy
    • Ubuntu offers mdadm for software RAID management
    • RAID 0 (Disk striping): RAID 0 splits data across any number of disks allowing higher data throughput. …
    • RAID 1 (Disk Mirroring): …
    • RAID 5 (Striping with parity): …
    • RAID 6 (Striping with double parity): …
    • RAID 10 (Striping + Mirroring):
    • Doesn’t seem to be much reason to use software RAID mdadm over ZFS?
    • Windows offers mirrored volume, similar to RAID 1
  • ZFS 
    • Supported natively in Ubuntu 18.04
    • FREENAS uses ZFS
    • ZFS Pools are equivalent of RAID levels

VIBRATIONS:

Tony – Peter Jones on Telegram writes

Really sorry, but didn’t enjoy this episode (317.5). Toyam spoke very passionately, but didn’t seem the right kind of distro to get a guest on about based on the MintCast audience. Anyway. I’m sure others loved it. And I can’t enjoy every episode!

I responded 

[In reply to Peter Jones]

I can see where you are coming from on the type of distro that Void is, but I think that the interview does highlight that it is possible to get involved with Linux projects, and the amount of work that the maintainers of Distros and the software we use on a day to day basis do so we get to use fantastic software. Mint might be an easier distro for a beginner to use but the work that has gone into it is as much if not more. Also it highlights that many of these distros and software development projects are running on very few people making it all happen. My feeling is even if you are not a coder is that there are many ways you can support, from beta testing and reporting bugs, to getting involved with documentation or creating wallpapers. If you feel that’s not possible then send the project a couple of dollars to help the cause and show your appreciation, or if you can’t even afford that, just send a message to say thank you for the work they are doing.

Peter replied

Hi, running a website and forum I understand the small team thing. Thanks for reading my feedback though and you make fair comments.

Stephen Heckler also commented on this episode:

I agree with Peter on this one, Void is pretty niche, and Toyam’s take on it even more so due to being a maintainer. mintCast has always been pretty broad within the Linux world and approachable

Leo – A discussion about Gentoo and compiling in Telegram

Londoner writes:

BTW, after I did Gentoo I installed Plasma base and some other packages. Compiling the Falkon browser (or rather qtwebengine) took 4 days on my i7 desktop! Thought it would never finish

Leo

The Juno Computer Gemini was finally reviewed by Oliver Kelly!

Oliver writes:

I reached out to them asked if I can help promote them as a company and write about them, they then offered to loan a laptop to review. They paid for shipping the laptop and collection. No pressure to write a positive review, to be fair I should have stated, but genuinely was a really nice laptop but £770 is steep for me for how it feels compared to my works xps , like I said , that’s the only other laptop I have to compare quality and feel to.

Writeup Link: https://0lzi.tk/episode-009-gemini15/

Tony H – Stan R – From tonight’s chat room

The new web page has really improved since the last show. It seemed that nothing was working. This time around things on web page seem much more usable to me.

Me – Yes Josh has put a lot of work in to get it working right

Stan – I had sent a number of complaining messages, then within a few hours it was either fixed or I realized what I was doing wrong.

Stan has also asked if it would be possible to put the show timer on the main page of the site.

CHECK THIS OUT:

• Leo – 
  • My Nextcloud writeup!
  • tldr – a terminal based help system. It can be used in place of “man” or “info” and provides mostly examples on how to use commands.
• Joe 
  • Mac? PC? You don’t have to choose..  RHKVM
• Tony – Keeping up with developments in the Linux podcast community, Destination Linux which is a great podcast has now morphed into the Destination Linux Network and includes all the current teams other work on YouTube and the recently started Linux for everyone podcast. They have a community forum and are planning to have a regular ‘Charity’ related to the open source movement where they will support others in our community. Check out their site for more information 
• How to Install Ubuntu Inside Windows 10 Using WUBI With UEFI Support
• Bo – https://www.linuxuprising.com/2019/09/rclone-browser-fork-with-fixes-and.html

ANNOUNCEMENTS:

Next episode will be Sunday October 6th, 2PM CDT, 7PM UTC

WRAP-UP:

• Joe – www.Tllts.org www.linuxlugcast.com MeWe [email protected]
• Bo – undercastnetwork.com
• Moss – Triad Bardic College, Peaceful Hippo, MeWe, music on Bandcamp and all over YouTube (search for Moss Bliss, ignore the young black South African gospel singer), [email protected], [email protected]
• Tony Hughes – HPR – http://hackerpublicradio.org/correspondents.php?hostid=338
  • Occasional Blog https://tony-hughes.blogspot.com/
  • Twitter @TonyH1212, [email protected] [email protected]

• Tony Watts – [email protected], Echoes of Savages (band)
• Josh – [email protected], [email protected], edublocks.org and @all_about_code on Twitter
• Leo – leochavez.org and @leochavez on Twitter, and Full Circle Weekly News

Before we leave, we want to make sure to acknowledge some of the people who make mintCast possible …

• Josh Lowe for all his work on the website and the livestream (We’ve had over 60 listeners today on the backup mixer platform and brought on some new listeners who’ve never heard of the podcast)
• Bytemark Hosting for hosting mintcast.org and our Mumble server
• Archive.org for hosting our audio files
• The Linux Mint development team for the fine distro we love to talk about <Thanks, Clem!>

Podcast: Play in new window | Download

Subscribe: RSS

• 
• 
• 
•