How to format a large external usb hard drive for use between both an OS X Mac and a Windows 10 PC

All versions of Windows since Windows Vista should be able to access a GUID drive. So, any modern computer since 2006 should be compatible. GUID doesn’t suffer from the restriction of a maximum partition size of 2TB, so if I have a larger hard drive than 2TB, it doesn’t require the multiple partitions that MBR does.

Here’s a good quote for other important features regarding GUID (GPT stands for GUID Partition Table).

“On an MBR disk, the partitioning and boot data is stored in one place. If this data is overwritten or corrupted, you’re in trouble. In contrast, GPT stores multiple copies of this data across the disk, so it’s much more robust and can recover if the data is corrupted. GPT also stores cyclic redundancy check (CRC) values to check that its data is intact — if the data is corrupted, GPT can notice the problem and attempt to recover the damaged data from another location on the disk. MBR had no way of knowing if its data was corrupted — you’d only see there was a problem when the boot process failed or your drive’s partitions vanished.”

exFAT was released in 2006 as well, but Microsoft added backwards-compatibility to previous Windows versions from before Vista. The main benefit to it is that it doesn’t have the file size restrictions of FAT32, so individual files with exFAT can be larger than 4GB each. It probably isn’t super important for smaller files, but it could be a necessity for people working on larger files like videos.

Below is a step-by-step procedure for formatting an External USB drive which can be used by both a Mac and a PC. This setup will utilize the newest, most fault-tolerant partition tables, and allows for the largest volume and file size capabilities. In my case I am formatting an 8TB Seagate Backup Plus+ USB 3.0 external HDD hard drive.

First, plug a new USB drive into a Mac:

  1. The Mac will automatically prompt if you want to use the drive as a Time Machine backup Disk – click “Don’t Use”
  2. Open Disk Utility
  3. On the left side of Disk Utility, under External, you should see your drive listed.
  4. Select the “highest-level” of the drive, not the partitions located underneath. In my case, Seagate Backup+ Desk Media.PC Mac External drive format (1)
  5. At the top of Disk Utility, click the “Erase” button.PC Mac External drive format (2)
  6. Name your disk, such as “JC-External”.
  7. Under “Format” drop-down menu, select “ExFAT”
  8. Under “Scheme” drop-down menu, select “GUID Partition Map”PC Mac External drive format (3)
  9. Click “Erase”
  10. Once the drive has been erased, again, Time Machine will prompt to use as a backup disk – select “Don’t Use”PC Mac External drive format (4)
  11. Click “Done”
  12. Your drive should now be listed under Devices in the Finder.PC Mac External drive format (5)
  13. Control-click or right-click on the device in the Finder, and click “Get Info”.  You can see that indeed it created an 8TB ExFAT Volume, but the Sharing and Permissions cannot be modified. Permissions can only be set if the drive is formatted with  “OS X Extended”. Also, notice that the Created/Modified dates may not be accurate, however, files and folders contained in the drive will display accurate modified dates/times.PC Mac External drive format (6)

 

Raspberry Pi 3 Model B 32 GB Setup and Config

Raspberry Pi 3 Model B 32GB Initial Setup and Configuration

So my first Raspberry Pi 3 Model B 32GB just arrived which is part of a “Canakit” on Amazon which can be found here. Below is a general setup and config step by step guide to get you started with your own Pi. I may follow up this post with the progress of my pie-in-the-sky project to build a miniature hyperloop “pod” and track, controlled by a stand-alone Raspberry Pi. But before I get into powering my Pi with an electrified rail, we need to get the Pi setup and configured. 

  1. Unbox, connect peripherals & power on (I swear, every video and tutorial spends too much time on this topic.) But I will say that the little machine is very quick and responsive, and I’m really happy with it’s performance so far!
  2. Initial power up. The Red LED powers on, and my monitor stays black, hmm, it’s not booting… Oh duh, it will help if I insert the included 32GB MicroSD. Important to note is the SD card does not click-eject or click-insert but is friction-based receptacle.
  3. Ok 2nd power up – it’s alive! Boots to the desktop and looks great – except the resolution. I used the Wifi config to get connected to my WAP easy. But it’s strange that I can’t ping anything. A quick search found this post, and the command:
     sudo chmod u+s /bin/ping

    With that command I can now ping out and about.

  4. Now I can go online with the default web-browser (Web 3.8.2) and find a fix for my monitor’s resolution. To set a raspberry pi 3 monitor resolution, follow the instructions in this post – which told me to first check my monitor’s capabilities with the commands:
    tvservice -d edid
    edidparser edid

    and then seeing that I do have a DMT monitor (group 2) and an hdmi mode 82, I edited /boot/config.txt with the new editor named Geany. I started geany with sudo so I could edit the protected file with the command:

    sudo geany
    

    Which brings up the Geany file editor running as sudo, and then I could open and browse to the File System > /boot/ directory where I could edit the file config.txt to include the following lines:

    hdmi_group=2
    hdmi_mode=82

    After a restart my monitor’s resolution is correct.

  5. So now to make sure my OS is up to date. By default the version I installed is
    Linux raspberrypi 4.4.9-v7+ #884 SMP Fri May 6 17:28:59 BST 2016 armv7l GNU/Linux
    

    So it is time to run an update. It’s so nice that the distro is debian-based and the familiar apt package management system is installed. No dist-upgrade necessary.

    sudo apt-get update
    sudo apt-get upgrade

    I’ll be following up soon with more findings and initial configuration, but for now I’ve got enough to be able to branch out and explore the world of Raspberry Pi!

OS X new domain migration – retaining user profile with terminal commands

If you’ve been tasked with the domain migration of a number of OS X El Capitan and Mavericks iMacs or MacBook Pro or Mac Air workstations, you may need to retain the user profiles. Normally, when unbinding, and then binding to a new domain, your user’s settings will be lost. You may be tempted to use Migration Assistant, but this usually requires copying the entire profile somewhere else which can take a long time and use a lot of disk space.

With this list of steps, you can use commands, scripting, and setting permissions and ownership of the user directories to perform the domain migration in-place.

Below is the sequence of commands and workflow step by step to migrate an OS X mac to a different domain. The key is to delete the sqlindex files, and prepare the user account for it’s new permissions. Please note this is a rough guide, but it will hopefully get you closer to migrating your macs so that the users keep their same profile. Let me know if this guide helps you in your domain migration and if you find any better solutions.

Tasks Commands
1 Login as admin user and list users  Terminal -> ls -alh /Users/
2 move domain User folders to .old sudo mv /Users/johndoe /Users/johndoe.old
3 Unbind Machine  Preferences->Accounts->Login Options->Network account server -> Directory utility ->Active directory->Unbind
4 Delete sqlindex files found in ls /var/db/dslocal/nodes/Default/ sudo rm -f /var/db/dslocal/nodes/Default/sqlindex
sudo rm -f /var/db/dslocal/nodes/Default/sqlindex-shm
sudo rm -f /var/db/dslocal/nodes/Default/sqlindex-wal
5 reboot sudo reboot
6 Bind to new domain Preferences->Accounts->Login Options->Network account server -> Directory utility ->Active directory->Unbind
7 reboot
8 login as user
9 Logout and login as admin
10 move User folder .new After you have logged in as the user under the new domain you need to move the newly created User home folder to johndoe.new and move the .old User folder to /Users/johndoe with the command:
sudo mv /Users/johndoe /Users/johndoe.new
11 move .old to new username sudo mv /Users/johndoe.old /Users/johndoe
12 Change ownership of user home folder sudo chown -R johndoe:”Domain\Domain Users” /Users/johndoe
13 logout as admin
14 Reboot
15 login as that user (johndoe)
16 Click on “Create a new keychain” much easier in El Capitan and Yosemite If “Create new Keychain” fails then goto Keychain Access –> Preferences –> Reset Default Keychain

Windows 10 Best Free Video Screen Capture Open Broadcaster Software Install Tutorial

This video demonstrates how to use Open Broadcaster Software for capturing your desktop into a record-able video. OBS is free software that you can also use to record your gaming sessions to never miss that moment during your computer game. Here is a good post on Reddit.com describing how to set up OBS for capturing your game with a 20 second “buffer” so you can show your frags on youtube or convert them to a gif. But for now, below I put together a tutorial on how to setup OBS to record your desktop screen and audio to make videos like the one here:

 

How to Setup User Account for Child on Windows 10 Tutorial

This video shows you how to setup a user account for your kid/child without creating an email address. Windows 10 wants you to use an email address to create a user account. When you try to setup an account, it asks you to use an existing email and if not, it tries to force you to sign up for an outlook.com account. This video shows you how to bypass the Windows User account setup and use Computer Management to create a normal/local user account on the computer for your kids.

Microsoft Bizspark – free business software for 3 years

If you’re thinking about which cloud service to use for a startup business, Microsoft just upped the ante with BizSpark.

Microsoft BizSpark https://www.microsoft.com/bizspark#start-two is really an amazing deal for business start-ups. If you wish you could get Microsoft software for free or for a huge discount check out their offer. BizSpark offers the following services and software for free for three years:

BizSpark gives startups 3 years of free stuff – software, services, tech support, and Azure cloud. Your startup qualifies if it is less than 5 years old, is privately held, and earns less than $1M annually. And at the end of your 3 years, you keep all the software you’ve downloaded – at no cost.

To expand on this service what you get with the Microsoft Bizspark details are the following:

Get up to $750 per month of FREE Azure cloud services for 3 years; that’s $150 per month each for up to 5 developers.

This potentially is a $27000 value!

Membership puts all Microsoft development and test software at your fingertips, including Azure, Windows, and Office 365 – for free. Plus, enjoy access to hundreds of free training classes, technical content, and 4 break-fix phone support incidents to help you on your journey.

It’s pretty amazing that BizSpark, in addition, also offers up to $120,000 worth of Azure credit.

Makes me want to go out and start a new business – hmm, maybe jasoncoltrin.com would qualify?

Moodle 2.4 – Recover uploaded course files from server even if corrupt

If for example, a new faculty member does not have the original files for an old moodle course, and the files uploaded to moodle the last semester are corrupt, you still can extract the files from a backup of the course with a tool I found here: DownloadAll

(DownloadAll.jar – cudos and thank you to OP Veronica Liesaputra !):

In order to extract all files from a course backup mbz file, please follow the guide below. Even if the files that were uploaded and are corrupt, and cannot be viewed/downloaded in moodle, you may be able to download the originals, repair the files, and then re-upload to moodle again:

  1. Go into the course where files are located (may be in an old location, even though current/copied course is valid with same corrupt files)
  2. Backup the course without user data. An mbz file will be generated. Copy this file to a Windows computer eg. c:\Users\jcoltrin\Desktop\moodlebackups\coursename.mbz
  3. Copy the linked jar script to the same folder
  4. Make sure 7zip is installed. Extract all the files in the mbz file to a directory with the same name just under the location of the mbz file. The script will not work unless you first extract all the files! For example, your folder structure will now look like this:

 

   Desktop\
     moodlebackups\
          DownloadAll.jar
          backup-moodle2-course-269-1509-bus302-va-20160114-1-28-nu.mbz
          backup-moodle2-course-269-1509-bus302-va-20160114-1-28-nu\
                   activities\
                   course\
                   files\
                         1b\
                              1b23cd2aaf9c0e8829407fc46c6b4b4c4f8d1a7c
                         3d\
                              3ddb69ac1035d0a51c4d1a3e767391d21f70c895
                   sections\
                   files.xml\

… This step may take a little bit of cut/paste as the mbz file has to be extracted twice with 7zip.

  1. Run the DownloadAll.jar script with the command java -jar DownloadAll.jar (PS C:\Users\jcoltrin\Desktop\moodlebackups> java -jar DownloadAll.jar)
  2. You’ll be prompted to enter the full path to the mbz file: “Enter the full path location to the Moodle’s backup (.mbz) file, e.g. C:/Users/Moodle/Backup/backup-moodle2-course-592-nu.mbz” – so in this case, I used: C:\Users\jcoltrin\Desktop\moodlebackups\backup-moodle2-course-269-1509-bus302-va-20160114-1028-nu.mbz

Output should return the names of the files that are recovered and start extracting (files even though errors occur):

       C:\Users\jcoltrin\Desktop\moodlebackups\backup-moodle2-course-269-1509-bus302-va-20160114-1028-nu created
       IOError :java.util.zip.ZipException: error in opening zip file
       Copy C:\Users\jcoltrin\Desktop\moodlebackups\backup-moodle2-course-269-1509-bus302-va-20160114-1028-nu\files\74\74ada129
       b720c6266753b54e5c5af614274bfcfe to C:\Users\jcoltrin\Desktop\moodlebackups\backup-moodle2-course-269-1509-bus302-va-201
       60114-1028-nu\Legal StructureLawsuits .pdf
       Copy C:\Users\jcoltrin\Desktop\moodlebackups\backup-moodle2-course-269-1509-bus302-va-20160114-1028-nu\files\6d\6db0919b
       e36df971bd7780defd8802ba119d4e80 to C:\Users\jcoltrin\Desktop\moodlebackups\backup-moodle2-course-269-1509-bus302-va-201
       60114-1028-nu\Sample-Negligence-Complaint.html

7. Once files have completed Extracting, gather the files and distribute as necessary.

8. In this case, many of the files that were corrupt in moodle had contained spaces in the filenames. I find that files uploaded to moodle work best with filenames that do not contain spaces or special characters. Also it’s smart to test opening each file in moodle first, to make sure repairs and download actions for each file work correctly.

Solved – Dell Optiplex 790 2nd HDD disk drive not found in Windows 10 Disk Management but BIOS sees the drive

disk

I have a Dell Optiplex I’m putting together for an IP Camera security system. The security cameras use a lot of disk space, so I connected a 2nd Seagate 2TB drive to the black SATA port labeled SATA1. The BIOS sees the drive, but when I logged into Windows 10 and looked in Disk Management, the drive wasn’t found. If the new hard drive isn’t in Disk Management, but the BIOS does see the drive, there’s something wrong with either the BIOS / SATA / RAID configuration, or there’s something wrong with the drive itself. I attempted to Scan for Hardware Changes in the Device Manager – no luck. I also went into Disk Management -> Action -> Rescan Disks -> no luck.

Here is the SATA Port layout:

SATA 0 (Blue) – Primary HDD 500GB Seagate ST3500413AS

SATA 1 (Black) – Secondary  HDD 2TB Seagate ST2000DM001-1CH164

SATA 2 (White) – CD / DVD

SATA 3 (White) – empty

I went to Dell’s support website, ran the System Detect (for some reason entering the Service Tag didn’t work) and then went to look at the available drivers. I was thinking of updating the BIOS from A09 to A18, but then noticed under Serial ATA there is a Seagate Firmware update named B765JC49.zip – unpacked is 2 folders, DOS and Windows, in the Windows folder is the file B7032100.exe – this is the file I installed. During the setup, the computer is restarted, and like most firmware utilities (I love these), you get a nice old-school 8-bit text interface with a resounding SUCCESS in big blue letters when it’s done.

So this was really all I did, and after flashing the Seagate Firmware, and logging into windows, immediately the drive was detected and prompted me to initialize and format the disk. So I did just that; initialized the drive as MBR, then changed the CD-ROM to drive letter E:, and formatted the new drive as NTFS on drive letter D: labelled as “Data”. Life is good again. Hopefully this post will be found by someone else having difficulty when their computer doesn’t see the new 2nd disk drive and save them a little time and frustration in the process.

 

Modern PHP development environment – Setup of Ansible, pycharm, sourcetree and workflow with bitbucket

When getting started with development with a cloud repository such as git, it may be a little daunting to decide how to get started. With some help from an associate, I put together a short simple guide to setup a development environment on OS X. I hope this information provides someone with a good start to development with bitbucket, version control, and PHP Development in conjunction with a cloud repository.

Bitbucket is similar to git, but allows free repos. We prefer to use bitbucket for a repository of code so that we can manage changes to our ubuntu servers and files. Bitbucket is the “Book of Truth” and will be the keeper of all files and things that are good. Ansible runs on a dedicated management ubuntu server and pushes out changes (playbooks) to either a single, a few, or all of our linux servers. Either way, with pull/push of data from our code repository, we can control what is deployed on our systems, an use our repo as our backup. If a server dies, we can setup a new system, and pull in the good data.

Setup

First, you need a bitbucket account and sign-on. Once signed on to https://bitbucket.org/brooksinstitute/ You should be able to create your first repo. You might want to create your own private repo for notes, configs etc. As mentioned earlier, bitbucket is where we keep our known-good source code, and changes to this should only be done from your own computer’s copy of the repo, and only changed with commits – more on this later.

sourcetree

Next on your local machine, download sourcetree https://www.sourcetreeapp.com/

Once downloaded and installed, tell sourcetree where your repos live at bitbucket (simple username/password login).

Next, SourceTree will ask you which remote repository you want to clone to your local machine. You want to clone the remote repos on bitbucket so that you can make changes to your local versions before you commit them back to bitbucket. If you work with a group of developers you will probably want someone to review your files before you commit. You should also “checkout” local copies within pycharm, if someone else will also be working on your local files.

pycharm

Now it’s time to install and configure pycharm Community Edition https://www.jetbrains.com/pycharm/ . Pycharm is a Development Environment (IDE) that provides code completion, nice pretty colors and integrates with VCS/Git to do versioning control of your local (cloned) repo. In Pycharm, you want to go to the File → Open menu, browse your local machine, and choose the root folder of the cloned repo of your choice. This will get you to the point where you can begin to edit files.

ansible

Ansible http://www.ansible.com/ is a management utility that helps you easily manage systems and deploy apps. Here is some introductory documentation http://docs.ansible.com/ansible/intro_getting_started.html. Ansible usually runs on a dedicated Admin server, and this is the server that issues commands or “playbooks”. Although your Admin server contains the ansible playbook files, we only want to make changes to the files linked to the bitbucket repo before we pull them into the Admin server and then execute the commands.

 

Vagrant

Vagrant https://www.vagrantup.com/ provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize productivity. First download, install, and run VirtualBox https://www.virtualbox.org/wiki/Downloads , then open a terminal, and startup a vagrant “box” with the following:

   $ vagrant init hashicorp/precise32
   $ vagrant up

Vagrant will download and install the ‘precise32’ “box”. And now, in virtualbox you will see the new virtual machine. Then next from the command line you can issue the command ‘vagrant ssh’ which will open a shell to your new precise32 vm. You can use this vm to test your configurations and playbooks against before you roll them out to your production servers.

Workflow

When you’ve changed something in your local (cloned) repo, and you want to have that become the “truth” on bitbucket, do the following:

  1. Open the file from your local repo in pycharm (double-click on the file icon in the menu tree)
  2. Edit the file
  3. When done editing, right-click on the file → Git → Commit file
  4. Now you want to push this edited file up to bitbucket. Review the code, make comments and then push.

 

How to create and shrink partitions in Windows 10 – video

I’ve posted a new video here that demonstrates how to shrink an existing partition with Disk Management and then create a new partition with the empty space in Windows 10.

Sometimes you’d like to have a separate partition for data or scratch drives to keep your data safe when you run backups or re-install an operating system or software. Other times you create new partitions to install a different operating system and do a “Dual Boot” with the extra partition. Let me know if you’d like to see a video or a post about dual-booting Windows 10 and Linux/Debian/Ubuntu.