Removing password protection from PDF files

Important note: this post wont help you if you have a PDF file and don’t know the password. This is for removing passwords on PDFs that you have legal access to, but don’t want to be password-protected any more.

A while ago, one of my employers started emailing payslips in PDF format. Now, I know there are many issues around accessibility with PDFs, but it works for me – I get a digital version of a document that looks exactly as the printed one would have. Except that someone decided email (even to a company-secured account) was not secure enough, and they password-protected the files. In theory, this stops another employee from opening my payslip. In practice, they used a known piece of personally identifiable information (PII).

Anyway, I wanted to keep a copy of the files on my own file storage. I can do this because, technically, they are not company data and they are (or at least should be) private to me. Indeed the company in question has since moved to a system that emails a link to a personal email account, inviting the employee to download their payslip from a portal.

I didn’t want the copies of the payslips that I held to be password protected. That meant I needed to remove those passwords.

QPDF

QPDF is a computer program, and associated library, for structural, content-preserving transformations on PDF files. It’s not for creating, viewing or converting PDF files.

One of the things it can do, is remove the password protection on a file. Remember, this is a file that I have legal access to, so removing the password protection is not a crime. I’m not hacking the file – in fact I need to know the password in order to remove it.

QPDF can do much more than remove passwords (for example I think I could use it to create new versions of a PDF file with just a subset of the pages), but this was what I needed to do.

A little side-note

This was the second time I performed this exercise. I first did it a few years ago, but only on the payslips I’d received up until that date. Later ones were still password-protected. I didn’t document my method the first time around though… so I had to work it all out again. This time I decided to write it down…

A little PowerShell Script

It looks like, the first time I ran this, I downloaded a Windows executable version of QPDF and either wrote, or more likely found, a PowerShell script to adapt. The script is called payslips.ps1 and looks like this:

$children = Get-ChildItem # Save files in a variable. Piping the rest of the script from Get-ChildItem in a single line was a bad idea
$children | ForEach-Object {
Write-Debug "Working on $_.Name"; #Doesn't actually display a lot
$fileName =[System.IO.Path]::GetFileNameWithoutExtension($_.Name); #Strip name, we will append "tmp"
$ext =[System.IO.Path]::GetExtension($_.Name);
$tempFile = $fileName + "tmp" + $ext; # Append "_tmp" Move-Item -Path $.Name -Destination $tempFile; #Move the file to a temporary location
..\qpdf.exe --password=AB123456C --decrypt $tempFile $_.Name; #Use qpdf to decrypt it, save in original location
#Remove-Item $tempFile #Remove temporary file
}

ABC123456C should be replaced with the actual password. Actually, it shouldn’t, because including credentials in code is sloppy security practice. There are better ways to pass the password, but I’m just converting 50 files as a one-off exercise, not building a repeatable business process. If you go on to use this in a business environment, please don’t do it this way!

Release notes

The script makes a temporary copy of each file, suffixed with _tmp but preserving the file extension.

If you run the script against the current folder, it will run against all files, not just PDFs. That means it will rename itself and all the QPDF files with _tmp. This will cause it to fail.

It looks like, when I ran this a few years ago, I used a files.txt file to control this behaviour. files.txt was just a list of filenames and is easily generated using the following command:

dir /b /a-d > files.txt

But, this time, I couldn’t see how to provide that as a parameter to QPDF, so I had to:

  1. Place all the files to be converted in a subfolder of the folder containing QPDF and my PowerShell script.
  2. Edit the payslips.ps1 script to refer to ..\qpdf.exe (i.e. qpdf.exe in the folder above the current one).
  3. Change directory into the subfolder.
  4. Run payslips.ps1 from the subfolder – i.e.:
..\payslips.ps1

This means it will only run against the files in the subfolder, and not against QPDF, the script, or anything else.

It doesn’t seem to remove the temporary files. I didn’t try to work out why. It had already created what I needed by then.

Featured image: author’s own

Tweaking audio and (webcam) video quality in Windows 10

This content is 4 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Back in the spring (whilst I was on Furlough Leave and had time for weeknotes), I wrote about some upgrades to my home office. The LED lights didn’t work out (battery life was too short – I need to find something that works from mains power) so they went back to Amazon but the Marantz MPM-1000U microphone has been excellent.

I’ve seen a few tweets and videos recently about using software to use a smartphone camera as a webcam. Why might you do that? Well, because many laptop webcams are a bit rubbish (like the one in my Apple MacBook) or poorly placed, giving an unflattering view from below.

I had a play with the Iriun webcam software recommended in this video from Kevin Stratverdt and it worked really well, with the phone on a tripod, giving a better angle of view.

Ultimately though, the Microsoft Surface Pro 6 that I use for work has a pretty decent webcam, and my Nokia 7 Plus was no better quality – all I was really gaining was a better camera position.

I do still have a challenge with lighting. My desk position means that I’m generally back-lit with a north-facing window to my left. Some fill-in light in front might help but I also wanted to adjust the settings on my webcam.

Microsoft Teams doesn’t let me do that – but the Camera app in Windows 10 does… as described at Ceofix, there is a “Pro mode” in the Windows 10 Camera app that allows the brightness to be adjusted. There are more options for still images (timer, zoom, white balance, sensitivity, shutter speed and brightness) but the brightness option for video let me tweak my settings a little.

The next challenge I had was with audio. Despite using the volume controls on the Surface Pro to knock the volume up to 100% whilst I was presenting over Teams earlier, everyone else on the call sounded very quiet. It turned out that 100% was not 100% – there is a Realtek Audio Console app on my PC which, as well as letting me adjust the speaker and microphone settings, including volume, balance, Dolby audio, sample rate and depth. Finding this revealed that my volume was actually no-where near 100% and I was quickly able to increase it to a level where I could hear my client and co-presenters!

Weeknote 22/2020: holidaying on the Costa del Great Ouse (plus password resets, cycling performance, video-conferencing equipment and status lights)

This content is 4 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

In the last few hours of 2019, my family planned our holiday. We thought we had it all sorted – fly to Barcelona, spend the weekend sight-seeing (including taking my football-mad son to Camp Nou) and then head up the coast for a few more days in the Costa Brava. Flights were booked, accomodation was sorted, trips were starting to get booked up.

We hadn’t counted on a global pandemic.

To be clear, I’m thankful that myself, my family and friends, and those around us are (so far) safe and well. By April, I didn’t much like the prospect of getting into a metal tube with 160+ strangers and flying for 3 hours in each direction. We’re also incredibly lucky to be able to access open countryside within a couple of hundred metres of our house, so daily exercise is still possible and enjoyable, with very few people around, most of the time.

I still took the week off work though. After cancelling my Easter break, it’s been a while since I took annual leave and even my Furlough period was not exactly relaxing, so I could do with a rest.

The weather has been glorious in the UK this week too, making me extra-glad we re-landscaped the garden last year and I’ve spent more than a few hours just chilling on our deck.

Unfortunately, we also got a taste of what it must be like to live in a tourist hotspot, as hundreds of visitors descended on our local river each day this weekend. It seems the Great Ouse at Olney has featured in a list of top places to swim in Britain, which was recently featured in The Times. It may sound NIMBYish, but please can they stay away until this crisis is over?

As for the holiday, hopefully, we’ll get the money refunded for the cancelled flights (if the airlines don’t fold first – I’m sure that if they refunded everyone they would be insolvent, which is my theory for why they are not increasing staff levels to process refunds more quickly); FC Barcelona contacted me weeks ago to extend my ticket and offer a refund if we can’t use it; and AirBnB had the money back in our account within days of us being forced to pull out due to cancelled flights.

(I did spend a few weeks effectively “playing chicken” with easyJet to see if they would cancel first, or if it would be us. An airline-cancelled flight can be refunded, but a consumer-cancelled flight would be lost, unless we managed to claim on travel insurance).

Even though I’ve had a week off, I’ve still been playing with tech. Some of my “projects” should soon have their own blog post (an Intel NUC for a new Zwift PC; migrating my wife’s personal email out of my Office 365 subscription to save me a licence; and taking a look at Veeam Backup for Office 365), whilst others get a brief mention below…

Please stop resetting user passwords every x days!

Regularly resetting passwords (unless a compromise is suspected) is an old way of thinking. Unfortunately, many organisations still make users change their password every few weeks. Mine came up for renewal this week and I struggled to come up with an acceptable, yet memorable passphrase. So, guess what? I wrote it down!

I use a password manager for most of my credentials but that doesn’t help with my Windows logon (before I’ve got to my browser). Biometric security like Windows Hello helps too (meaning I rarely use the password, but am even less likely to remember it when needed).

Here’s the National Cyber Security Centre (@NCSC)’s password guidance infographic (used with permission) and the associated password guidance:

This list of 100,000 commonly used passwords that will get blocked by some systems may also be useful – from Troy Hunt (@TroyHunt) but provided to me by my colleague Gavin Ashton (@gvnshtn).

Performance analysis for cyclists, by cyclists

I’ve been watching with interest as my occasional cycling buddy (and now Azure MVP) James Randall (@AzureTrenches) has been teasing development on his new cycling performance platform side project. This week he opened it up for early access and I’ve started to road test it… it looks really promising and I’m super impressed that James created this. Check it out at For Cyclists By Cyclists.

Podcasting/video conferencing upgrades in my home office

With video conferencing switching from something-I-use-for-internal-calls to something-I-use-to-deliver-consulting-engagements, I decided to upgrade the microphone and lighting in my home office. After seeking some advice from those who know about such things (thanks Matt Ballantine/@ballantine70 and the WB-40 Podcast WhatsApp group), I purchased a Marantz MPM-1000U microphone, boom arm, shock mount, and a cheap rechargeable LED photography light with tripod.

It’s early days yet but initial testing suggests that the microphone is excellent (although the supplied USB A-B cable is too short for practical use). I had also considered the Blue Yeti/Raspberry but it seems to have been discontinued.

As for the photo lighting, it should be just enough to illuminate my face as the north-facing window to my left often leaves me silhouetted on calls.

Smart lighting to match my Microsoft Teams presence

I haven’t watched the Microsoft Build conference presentations yet, but I heard that Scott Hanselman (@shanselman) featured Isaac Levin (@isaacrlevin)’s PresenceLight app to change the lighting according to his Windows Theme. The app can also be used to change Hue or LIFX lighting along with Teams presence status, so that’s in place now outside my home office.

It’s not the first time I’ve tried something like this:

One particularly useful feature is that I can be logged in to one tenant with the PresenceLight app and another in Microsoft Teams on the same PC – that means that I can control my status with my personal persona so I may be available to family but not to colleagues (or vice versa).

One more thing…

It may not be tech-related, but I also learned the differences between wheat and barley this week. After posting this image on Instagram, Twitter was quick to correct me:

As we’re at the end of May, that’s almost certainly not wheat…

Weeknote No 2: Thule bike carriers; Microsoft #FutureDecoded; and a new iPhone (Week 44, 2017)

This content is 7 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

After some positive feedback on last week’s newsletter-style blog, I’ve decided to keep going with the format for at least another week.

So, please indulge me in a little narcissism as I write about a week in the world of Mark… although this post is a little late as it’s now the following Monday (I ran out of weekend…)

New bike carriers

Last week I wrote about racing cyclocross with my son. I have a 4-bike carrier that fits on a towball on my car but it’s just a cheap one from Halfords and, to be honest, it’s not that great. I’ve been considering getting a roof mounted system for when I have just a couple of bikes (i.e. not the whole family’s) and, I decided to buy a good product this time (buy cheap, buy twice…).

That meant a Thule system – and their website helped me to work out which parts to buy but I was still looking to do better than recommended retail price. After failing to land a couple of Thule ProRide carriers on eBay, I bought the whole setup from RoofRacks.co.uk, including matching locks as standard and free standard delivery.

One thing I thought long and hard on was whether to go for silver or black finish (my car has black roof rails). In the end. I decided on silver – the 10% premium for black parts is simply not worth it – especially as the silver rails/racks have some black components.

Survey marks

Whilst walking in Dorset last week, I spotted a strange disc bolted to a pavement, with the words Survey Mark on it. I asked Ordnance Survey if they knew what it is and they responded to say it’s a “historical bolt style benchmark” – a legacy system for recording the height above sea level.

Back to work (highlights)

After last week’s holiday, it was back to work this week – with a bang. My employer, risual, was headline sponsoring Microsoft’s Future Decoded event – which meant a couple of full-on (but enjoyable) days at London ExCeL in a mixture of stand duty (chatting to delegates, capturing potential sales leads), presenting (4 short sessions on digital transformation) – albeit in a theatre “room” at the side of the main exhibition hall (so not the best environment) – and joining the keynote sessions (though I missed all of the breakout sessions). Added to UK Azure User Group events on Monday and Tuesday evenings, it was a very busy few days!

I really enjoyed the presenting opportunity – I’d like to do more if I get the chance, though I do prefer to create my own content (rather than presenting material created for me). I also saw some pretty cool presentations that I hope will result in some blog posts of their own – particularly the ones around Quantum Computing and DNA Storage.

Accessibility

Hobbling around with a twisted ankle (after last week’s unexpected fall into the sea) has given me a little insight into what it’s like to have limited mobility. I’ve still walked, but more slowly than usual – and not the distances I’d normally expect to cover. No cycling, running or circuits this week either…

The closing keynote at Future Decoded had a major focus on inclusivity and accessibility – including the surprising statistic that 1 billion people in the world are disabled in one way or another (hidden or visible).

Every one of us has reduced ability from time to time – not just people who are disabled. That may be permanent, temporary (as in my case) or situational (such as when holding a child whilst on a phone call). Assistive technology is something that we can all use to make the most of our senses and get the best use of time – the most important thing we have!

iPhones…

Readers of last week’s post may remember that I fell into the sea, with an Apple Watch Series 3 on my wrist (water resistant) and iPhone 6s in my pocket (not water resistant). As well as the discomfort from the twisted ankle, that’s turned out to be quite an expensive slip…

After a couple of days drying out, my phone was working (sort of), with notifications (and even a phone call) on my Apple Watch – and my computers could “see” the iPhone. But the screen wasn’t working so I couldn’t unlock it.

The damage to the phone was covered on my home contents policy as accidental damage but it was going to take a couple of weeks for the insurers to get their agent to collect, assess and then potentially return a repaired device to me. I don’t want a repaired device. Water damage leads to all sorts of longer-term issues, particularly when combined with corrosion, so they agreed to replace my phone if Apple would certify that the device was beyond economic repair due to liquid damage.  After seeing the bright red liquid damage indicator, Apple was happy to do that. Unfortunately, they valued my iPhone 6S at £299 – apparently the replacement price for an upgrade. Take off £100 excess and I had £199 in the bank but no working phone.

I’d only been saying how expensive the new iPhone is and how I’d keep mine for a bit longer the day before I trashed my old one… now I’m paying for that expensive iPhone 8 Plus over 20 months, with interest-free finance through the Apple Upgrade Programme (AUP… or “ay-up” as the staff referred to it… I thought I’d suddenly been transported to Yorkshire). It also gives me the option to exchange for a new phone in a year’s time (iPhone 8S, 9 or 11 or whatever the next one is called), and it includes 2 years’ Apple Care. Let’s hope the camera is as good as I was led to believe by some of my friends (that’s why I got the plus, and why I got the 256GB version).

Unfortunately, iCloud wasn’t backing up as much as I hoped and a restore to my new phone was a little underwhelming. I had backed up my photos manually but there were a few I hadn’t got, and I had some expenses I really wanted to click “upload” on. I searched the ‘net for a local Apple repair specialist to see how much a new screen might cost and found Northampton Apple Repair, who helped me out with a temporary screen and battery so I could take a full iTunes backup of my phone. Having seen the inside of my phone (lots of salt), they were amazed it even booted.

I also learned that:

Other stuff

The Carrot Weather app has an AR mode and it’s pretty cool:

This is what an Azure Stack looks like. Yes, it’s just a (mostly empty) rack of 1U servers and some very clever software:

This is what a Tesla looks like under the covers:

The Apple 3.5mm to Lightning audio converter is likely to get lost. Maybe leave it permanently attached to a set of headphones (via @timbo_baggins)?

After a few months of using Todoist Premium for free (thanks to discount codes), I’ve signed up for a year… it must be good because I suffer from subscription fatigue and am trying to avoid adding to the pile of products that I use for “less than the price of a cup of coffee” a day/week/month/whatever. They add up to a lot of coffee…

Weekend

No cycling for me this weekend but a good opportunity to get together with friends for a bit to eat and drink, followed by fireworks. After reading some night-time photography suggestions from Apple I downloaded a different iOS camera app ( Procam 5) but didn’t really get the opportunity to try it out before the live display…

Taking photos of fireworks is never easy – particularly on a smartphone. I’m quite pleased with some of the firework pics I took last summer though…

2017-07-08 22.36.21.jpg

I fitted the new roof bars on Sunday – they look pretty good. I tested the bike carriers too but took them off until I need to use them.

The instructions are OK, once you get your head around them, but this video helped a lot:

Wrap-up

I’ll be back with more next week – probably a little-less Apple-centric but I need to balance out this week’s Microsoft-centric tweeting, I guess! ^MW

A newsletter? Weeknote? Blogletter? Issue No 1 (Week 43, 2017)

This content is 7 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Inspired by David Hughes (@DavidHughes) and Christian Payne (@Documentally), a few weeks ago, I ran a Twitter poll to see if anyone would be interested in a newsletter of some of the stuff I’ve been up to. The responses were mixed, but some went along the lines of “the email format doesn’t resonate with me” and “I like reading what you’ve been up to on your blog”. My blog has been falling by the wayside in recent months and I do want to write more, so I’ve decided to write a weekly (ish) newsletter here instead. In between, I’ll stick write the usual tech-inspired stuff but this will be more eclectic. Matt Ballantine (@ballantine70) does something similar with his weeknotes – but he must be incredibly disciplined to get them out every Friday. I spend Fridays trying to end my week.

So, here goes for issue 1. I’m still not sure what this thing should be called?

A week off

I’ve just had a week off work. I needed it. My previous blog post describes some of the challenges I’ve had lately and I really needed to decompress. After the initial weekend madness (just like every weekend), the first half of the week was spent at home, mostly sorting stuff out (more on that later), then a few days away with my family…

The weekend before…

My eldest son has started competing in the Central Cyclocross League and I’ve been joining in the novice races whilst he races in the Under 14s (both races take place on the same course at the same time).

I seriously considered not racing last week after a very hard practice lap but then my son instructed me to “put your numbers on and race your bike”. Oh, OK then!

I’m reasonably fit for long distance stuff (I recently completed the rather hilly inaugural Velo Birmingham 100 mile sportive) and my Caveman Conditioning (circuits) a couple of times a week help with general fitness but cyclocross is something else. Particularly when you’re using a mountain bike because your son is riding his CX bike (how inconsiderate!). I think it may be time for an n+1. Certainly if we do this again next season!

Unfortunately, being ignored in the LBS doesn’t leave a very good feeling. Being ignored on social media after sending the tweet even less so…

Shopping

I don’t often wear a suit for work these days – but there are occasions where it’s still expected (first meetings, particular customers, etc.). I’ve been putting off buying a new suit for a while because a) there are two in the wardrobe that I really should slim down into b) I’d rather spend the money elsewhere. This week I gave in and bought something new.

I took one of my sons with me and he happily browsed the John Lewis technology department whilst I was suit shopping. He thinks I spent a lot of money though and suggested I get a blazer with some M&S trousers like his school uniform for a fraction of the price! Welcome to the world of work, son!

Whilst he was browsing the technology, I spotted this:

The Windows Premium collection appears to be Windows 10, running on a selection of higher-end PCs (Dell XPS 13, HP Spectre, etc.). First time I’d heard of it though…

Administration

I spent a good chunk of my week off working through an administration backlog at home. Ultimately that results in a lot of scanning (on my Canon ImageFormula P-215 desktop scanner), some shredding and a little bit of filing (for those few documents that I do retain in paper form).

After hunting around for PDF editing tools (ideally command line) to remove some pages I didn’t need inside some existing PDF files, I found this comment on the MacRumors forums:

“Preview does all of this quite well, fyi.”

Sure enough: open the PDF in MacOS Preview; delete the extra pages; save. Job done.

Karting, photography and train travel

My youngest son wanted to go to a friend’s go-karting party this week whilst my wife and eldest were heading down to Dorset for a few days. No problem, he could stay at home with me whilst I did some of my admin and then we’d follow on by train.

The karting inspired me to get my Nikon D700 out again. It may be big and heavy but I love the control of the DLSR experience and the results. I’ve tried some pro apps on my iPhone (like 645 Pro) but it’s just not the same!

_DSC7044

Afterwards, the train journey to Dorset gave my son and I a mini-adventure (bus, train, tube, another train) to join the rest of the family – and with a Family and Friends railcard it was less than £30!

Walking

Last Friday was a gorgeous day – almost no wind and bright sunshine didn’t seem like late-October! My family took the chance to go for a walk along the South West Coastal Path from Swanage to Studland (for a pub lunch).

Afterwards, I walked back with one of my sons – and what a treat that was! Glorious views and late-afternoon sunlight meant lots of photo stops but it was certainly my favourite part of the walk!

2017-10-27 16.43.07

2017-10-27 16.58.00

2017-10-27 17.08.54

On the beach

Saturday’s weather was less impressive but, after lunch at our favourite Swanage coffee shop (Java), coincidentally located next to my favourite Swanage restaurant (Chilled Red, where my wife and I had eaten the night before), we took the boys to the beach. They were happy with their wetsuits to keep the cold at bay whilst they played but I decided to stay dry. At least that was the plan.

I was walking out on one of the groynes to take a picture of the boys, when I found that walking boot soles have almost no grip once they meet wet wood and, faced with the choice of falling face-first (or probably chest-first) onto  a large wooden beam or throwing myself towards the sea, I chose the latter… managing to twist my ankle on the way, and then realising that my wallet and my iPhone were in my pockets.

I’m hoping that the phone will be covered on the household building and contents insurance – we have accidental damage cover and I’ll be making that call tomorrow… otherwise I could be getting an iPhone 8+ sooner than planned!

In the meantime, I’ve found out a lot about the water resistance of various Apple products:

Zwift and Android

My son fancied having a go on my Tacx Vortex trainer today, so we tried to get it working with Zwift for him.

Normally, I use the iOS app on my iPhone but, as that’s still drying out, it wasn’t an option. Zwift is currently available for Windows, MacOS and iOS but not (yet) Android so we went back to my original Windows PC-based setup with Zwift Mobile Link as a Bluetooth bridge. After spending a lot of time trying to get it working this afternoon with my son’s Android phone, it seems that I may need to update the firmware on my trainer for it to be recognised as a controllable trainer via the Android version of Zwift Mobile Link and Bluetooth LE (currently they only see it as a power meter and cadence sensor).

Wrap-up

That’s about it for this week… let me know what you think of the whatever-this-is (newsletter? blog post? something else?) and I’ll think about writing another one next week.

Using a VPN to watch ITV content outside the UK

This content is 7 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Those who follow me on Twitter (@markwilsonit) will probably be aware that I recently spent some time in mainland Europe – travelling through France, Germany and Switzerland with my family. You’ll probably also be aware that one of my hobbies is road cycling – and that I like to watch the highlights from the three Grand Tours (Giro d’Italia, Tour de France and Vuelta a España) and from the Tour of Britain. With the Vuelta in full swing as my holiday started, I wanted to make sure I could still catch the highlights on ITV4!

Even with the new EU mobile roaming arrangements that mean I can use my mobile data allowance in other EU countries, I didn’t expect to be able to stream content reliably, so I took out a subscription to ITV Hub+, allowing me to download ITV programmes with the ITV Hub app (on Wi-Fi) and play back later, without ads. This worked brilliantly on the ferry to France but not so well once I was in my Paris hotel room, where the app detected I was outside the UK and denied access to content with a variety of error messages:

ITV Hub download error outside the UK ITV Hub download error outside the UK ITV Hub download error outside the UK

I was pretty annoyed – after all, there was no mention of UK-only coverage when I subscribed to the ITV Hub+ and the ITV website says:

“Where can I use a Hub+ subscription?

As long as you’re signed into your account, you’ll be able to use your Hub+ subscription almost anywhere. Watch ad-free telly on our website, download and catch up on the go on your mobile or tablet, or binge on your favourite shows with no interruptions on your Smart TV!”

but I did find the limitation in their troubleshooting guide later:

I am abroad and can’t watch videos
The ITV Hub is only available within the UK as we don’t hold international rights for all of our shows. If you’re lucky enough to be on holiday or you live abroad, you won’t be able to watch ITV Hub until you return to the UK”

After a bit of a rant on Twitter (no response from ITV, of course), I thought about using a VPN (and @JFDuncan suggested Plex).

Unfortunately, my own VPN back to my NAS didn’t work (on reflection, L2TP/IPSec was not the best choice of transport – as @GarryMartin pointed out when I originally set it up) and I was nervous about using a third party service until Justin Barker (@JustinBarker77) suggested TunnelBear:

Recommendations are always good. And TunnelBear seemed more legitimate than some of the sites I found…

At first, I didn’t have much luck – even after following TunnelBear’s troubleshooting advice for accessing content. 24 hours later though, something had cleared (maybe I had a different IP address, maybe it was something on my iPhone) and ITV Hub+ worked flawlessly over hotel Wi-Fi and a VPN back to the UK. I could download my cycling highlights for later playback and the VPN tunnel even seemed to improve the Holiday Inn Wi-Fi reliability – possibly due to QoS restrictions prioritising potential business traffic (VPN) over leisure (downloading videos)!

I did have some challenges with playback – so I put the iPhone into Airplane Mode before watching content, just in case the ITV Hub app detected I was outside the UK again, but each time I wanted to download over the next few days I enabled the VPN and all was good. I also subscribed to TunnelBear for a month’s worth of unlimited data allowance (I soon chewed through the 1GB I got for tweeting about the service!).

Hopefully, this information will help someone else who’s frustrated by paying for a download service and then finding it doesn’t work outside the UK…

Downloading multiple YouTube videos for offline playback

This content is 7 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

A few weeks ago, I wrote about a nifty utility called youtube-dl that can be used to download content from YouTube for offline playback (indeed, I’m writing this on the last day of my holidays, having played precisely none of the content I downloaded to watch whilst I was away!).

In the original post, I suggested giving youtube-dl the URL for a playlist to download all videos in the playlist. It’s also smart in that if it detects any videos that are already present in the folder, it will skip them – e.g.:

[download] Cloud Tech 10 – 3rd July 2017 – Azure Machine Learning, Jenkins, Petya detection and more-ymKSGTR55LQ.mp4 has already been downloaded

But what if you want to download lots of videos that are unrelated – or just certain videos from a large list? In my case, I wanted to download a bunch of recent videos from the Global Cycling Network (GCN) – a YouTube channel that I often watch but which has thousands of videos – I certainly didn’t want to download the entire playlist!

Instead, create a file with the download commands for the individual videos, e.g.:

youtube-dl -f 22 https://www.youtube.com/watch?v=5RsFWlvJjOg
youtube-dl -f 22 https://www.youtube.com/watch?v=O7FxZ1kFIW0
youtube-dl -f 22 https://www.youtube.com/watch?v=iOaeo3_E8R4

Rather than sitting at the terminal, running each one and waiting, save the file with a .sh extension (assuming a Unix-based OS – like MacOS) and then kick them off at once.

My file was called dl-gcn.sh but it’s no good running that from the Terminal – bash will complain.

bash: dl-gcn.sh: command not found

Instead, prefix with sh as follows:

sh dl-gcn.sh

and the downloads should run through in serial fashion, whilst you get on with something more interesting…

Combining GPX files for Strava

This content is 7 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

This morning was spent on my bike… as was a fair chunk of this afternoon… as is a fair chunk of many summer weekends, much to Mrs W’s disappointment.

My friend Andy and I put in 60 miles in the sunshine, on a big loop around Milton Keynes. It seems my route planning is pretty spot on, as it was almost the exact opposite of a charity ride going the other way around (we passed the same riders twice!). Unfortunately, my ability to “press the start button on my Garmin cycle computer” is clearly less good – I was about a mile from home and heading out of town when I realised I’d forgotten to start tracking my ride!

My OCD can’t cope with this. It would be able to cope with turning around, going back up the hill, starting the computer and starting the ride again – but not with some missing kilometres in my ride data! Luckily, Andy was also riding with a Garmin bike computer. Even though he’d also forgotten to start his, he was wearing a Garmin watch too – so I could combine his data and mine (we’d ridden side by side for the first part of the ride…).

I’ve blogged before about GPS Track Editor, which is a fantastic piece of free software. Using this, I could edit Andy’s data to just the part I had missing, then combine it with mine and merge the two tracks (the short gap doesn’t matter – Strava will straight-line the route between the two points). I also tried merging the files with a tool from gotoes.org – unfortunately, that ended up with a ride that was effectively double the length of what we rode (two loops). it would probably have worked with my edited files but I could also merge them in the GPS Track Editor…

Combining tracks in GPS Track Editor

I then deleted the original (short) ride from Strava and re-uploaded. Sorted.

Just one thing to sort out – all of the PRs I got on today’s ride (and there were a few) were recorded as second places by the second upload. No worries – Strava has a “refresh my achievements” tool. which sorted out that particular issue. Now my ride has the complete distance… and my achievements are correct too…

Installing youtube-dl on a Mac to watch YouTube videos when working offline

This content is 7 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

My work pattern at the moment means that I’m spending a lot of time travelling on trains up and down the country (in fact, as this post is published, I’ll be somewhere between Bedford and Sheffield). A combination of fatigue and motion sickness means that this isn’t always a good opportunity to work on the train but it is potentially an opportunity to listen to podcasts or, unlike when I’m driving, to watch some videos. Unfortunately, travelling at 125 miles an hour with a varying quality of 4G data signal doesn’t always lend itself well to streaming from YouTube, etc.

That’s where youtube-dl comes in – I can download videos to my MacBook before I leave home, and watch at my leisure on the train. So, how do you get started?

Well, the Mac App Store website helped me. Following advice there, I issued two commands in Terminal to first install the HomeBrew package manager for MacOS, and then to install youtube-dl:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null

brew install youtube-dl

So, with youtube-dl installed, how do I use it? The youtube-dl readme.md file has lots of information but it’s not exactly easy to digest.

I found that:

youtube-dl -F youtubeurl

would give me a list of available video formats for a given URL and reading about YouTube media types led me to the very important number 22 for MP4 video at 720p with H.264 encoding and AAC audio. That should play on a wide variety of devices (including Quicktime on my Mac).

Next, to download:

youtube-dl -f 22 https://www.youtube.com/channel/UCz7bkEsygaEKpim0wu_JaUQ

(this URL is the CloudTechTV channel).

That command brought down all of the videos in the channel but I can also download individual episodes, for example:

youtube-dl -f 22 https://www.youtube.com/watch?v=ymKSGTR55LQ
youtube-dl
I can do something similar for other YouTube videos/channels (and even for some other video services) and build a library of videos to watch on my journeys, without needing to worry about an Internet connection.

“You need to work less”. Musings on finding the elusive work-life balance

This content is 8 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

“You need to work less”, said David Hughes (@davidhughes) as we were discussing why I carried a power supply with my Surface Pro. This was in response to my observation that the device will get me through the work day but not through travel at each end as well.

“Actually, you have a point”, I thought. You see, weekdays are pretty much devoted to work and pseudo-work (blogging, social media, keeping up to date with tech, etc.) – except for meals, sleep, the couple of hours a week spent exercising, and a bit of TV in the evening.

David commented that he reads – rather than working – on the train (I tweet and email but really should read more). And when I asked how he organises his day, he introduced me to ToDoIst. It seems that having a task list is one thing but having a task list that can work for you is something else.

Today was different. I knew I wanted to get a blog post out this morning, finish writing a white paper, and find time to break and meet with David in my favourite coffee shop. I’m terrible at getting up on working-from-home days (more typically working well into the evening instead) but I had managed to be at my desk by 7am and that meant that when I left the house mid-morning I’d already got half a day’s work in. For once, I’d managed some semblance of work-life balance. The afternoon was still pretty tough and I’m still working as we approach 7pm (my over-caffeinated state wasn’t good for writing!) but I met my objectives for the day.

Now I’ve added ToDoIst to my workflow I’m hoping to be more focused, to wrap up each day and set priorities for the next. I need to stop trying to squeeze as much as I can into an ever-more-frantic existence and to be ruthless with what can and can’t be achieved. Time will tell how successful I am, but it feels better already.