Jekyll2021-04-24T16:12:34+08:00https://stouty.xyz/feed.xmlstouty.xyzJames Stout. iOS. Web developer. Tech, crypto, comedy, Hong Kong, hcafc, smashing telly and, of course, the weather.James StoutAutomate Docker Image Updates on Synology NAS2021-03-31T07:03:25+08:002021-03-31T07:03:25+08:00https://stouty.xyz/synology/docker/2021/03/31/automate-docker-updates<p>I can’t believe it’s taken me this long to find <a href="https://github.com/containrrr/watchtower">Watchtower</a><sup id="fnref:fn-portainer" role="doc-noteref"><a href="#fn:fn-portainer" class="footnote" rel="footnote">1</a></sup> for Docker.</p>
<p>My previous process for updating containers to the latest images was a weekly job that ran a script like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">logExt</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">date</span> +%Y-%m-%d<span class="si">)</span><span class="s2">.log"</span>
<span class="nv">LOG_FILE</span><span class="o">=</span><span class="s2">"/var/services/homes/stouty/logs/</span><span class="si">$(</span><span class="nb">basename</span> <span class="s2">"</span><span class="nv">$0</span><span class="s2">"</span><span class="si">)</span><span class="s2">.</span><span class="nv">$logExt</span><span class="s2">"</span>
<span class="c"># get list of all image:rev</span>
<span class="c"># not mariadb|redis as they cause issues</span>
<span class="k">for </span>image <span class="k">in</span> <span class="si">$(</span>docker ps <span class="nt">--all</span> | <span class="nb">grep</span> <span class="nt">-Ev</span> <span class="s1">'mariadb|redis|ID'</span> | <span class="nb">awk</span> <span class="s1">'{ print $2 }'</span> | <span class="nb">grep</span> <span class="nt">-v</span> <span class="s1">'\d+'</span><span class="si">)</span><span class="p">;</span> <span class="k">do
</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$image</span><span class="s2">"</span>
docker pull <span class="s2">"</span><span class="nv">$image</span><span class="s2">"</span> | <span class="nb">tee</span> <span class="nt">-a</span> <span class="s2">"</span><span class="nv">$LOG_FILE</span><span class="s2">"</span>
<span class="k">done
if</span> <span class="o">[</span> <span class="s2">"</span><span class="si">$(</span><span class="nb">grep</span> <span class="nt">-c</span> newer <span class="s2">"</span><span class="nv">$LOG_FILE</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span> <span class="nt">-gt</span> 0 <span class="o">]</span><span class="p">;</span>
<span class="k">then</span>
<span class="c"># email me the list of new images</span>
<span class="k">fi</span>
</code></pre></div></div>
<p>Then I’d go into the the <a href="https://www.synology.com/en-global/dsm/packages/Docker">Synology Docker UI</a>, stop all the updated images, export their settings, run a docker rm command then re-import the settings. Quite a faff, and if I wanted to update MariaDB, even more of a faff as is has linked containers.</p>
<p>Enter <a href="https://containrrr.dev/watchtower/introduction/">Watchtower</a>:</p>
<blockquote>
<p>Watchtower is an application that will monitor your running Docker containers and watch for changes to the images that those containers were originally started from. If watchtower detects that an image has changed, it will automatically restart the container using the new image.</p>
<p>Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.</p>
</blockquote>
<p>And the <a href="https://containrrr.dev/watchtower/linked-containers/">coolest thing</a>?</p>
<blockquote>
<p>Watchtower will detect if there are links between any of the running containers and ensures that things are stopped/started in a way that won’t break any of the links. If an update is detected for one of the dependencies in a group of linked containers, watchtower will stop and start all of the containers in the correct order so that the application comes back up correctly.</p>
</blockquote>
<p>Alas you can’t use the Synology Docker UI to start to container<sup id="fnref:fn-sock" role="doc-noteref"><a href="#fn:fn-sock" class="footnote" rel="footnote">2</a></sup>, so you have to drop to the command line:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-d</span> <span class="se">\</span>
<span class="nt">--name</span> watchtower <span class="se">\</span>
<span class="nt">-v</span> /var/run/docker.sock:/var/run/docker.sock <span class="se">\</span>
containrrr/watchtower
</code></pre></div></div>
<p>Read all the <a href="https://containrrr.dev/watchtower/">docs</a> to configure how you want things. I ended up with a env file, a bootup job, and this docker command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-d</span> <span class="se">\</span>
<span class="nt">--name</span> watchtower <span class="se">\</span>
<span class="nt">--env-file</span> /volume1/homes/james/.docker/env.list <span class="se">\</span>
<span class="nt">-v</span> /var/run/docker.sock:/var/run/docker.sock <span class="se">\</span>
containrrr/watchtower
</code></pre></div></div>
<p>My env file:</p>
<div class="language-ini highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="py">TZ</span><span class="p">=</span><span class="s">Asia/Hong_Kong</span>
<span class="py">WATCHTOWER_DEBUG</span><span class="p">=</span><span class="s">false</span>
<span class="py">WATCHTOWER_POLL_INTERVAL</span><span class="p">=</span><span class="s">60</span>
<span class="py">WATCHTOWER_NOTIFICATION_URL</span><span class="p">=</span><span class="s">pushover://shoutrrr:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@xxxxxxxxxxxxxxxxxxxxxxxx/?devices=iphone telegram://17849ABCD27658:ccccccccccdsfdsfsfsf@telegram?channels=my_channel</span>
<span class="py">WATCHTOWER_ROLLING_RESTART</span><span class="p">=</span><span class="s">true</span>
<span class="py">WATCHTOWER_CLEANUP</span><span class="p">=</span><span class="s">true</span>
</code></pre></div></div>
<p>The notification options are numerous:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">email</code> to send notifications via electronic mail.</li>
<li><code class="language-plaintext highlighter-rouge">slack</code> to send notifications through a <a href="https://slack.com/intl/en-hk/">Slack</a> webhook.</li>
<li><code class="language-plaintext highlighter-rouge">msteams</code> to send notifications via <a href="https://www.microsoft.com/en-US/microsoft-teams/group-chat-software">MSTeams</a> webhook.</li>
<li><code class="language-plaintext highlighter-rouge">gotify</code> to send notifications via <a href="https://gotify.net/">Gotify</a>.</li>
</ul>
<p>I’m trying out <a href="https://containrrr.dev/shoutrrr/">Shoutrrr</a> with <a href="https://pushover.net/">Pushover</a> and <a href="https://telegram.org/">Telegram</a> <a href="https://core.telegram.org/bots">bots</a>.</p>
<p>Let’s see if it works in a week or so…</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:fn-portainer" role="doc-endnote">
<p>Yes, I tried <a href="https://www.portainer.io/products/community-edition">Portainer</a>, but it wasn’t for me. <a href="#fnref:fn-portainer" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:fn-sock" role="doc-endnote">
<p>Check out this Reddit <a href="https://www.reddit.com/r/synology/comments/bat67m/docker_impossible_to_make_portainer_to_access/eke9rpf/">thread</a>. <a href="#fnref:fn-sock" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>James StoutAutomatically update Docker images running on Synology NASCountly For iOS and Web Analytics2020-05-18T17:45:32+08:002020-05-18T17:45:32+08:00https://stouty.xyz/ios/2020/05/18/countly<p>I previously <a href="https://stouty.xyz/ios/2020/05/15/firebase-crashlytics/">mentioned</a> that I wasn’t too keen on all the things Firebase Crashlytics installed and would look at <a href="https://count.ly/product">Countly</a> as an alternative.</p>
<p>Countly is impressive. They have loads of <a href="https://support.count.ly/hc/en-us/categories/360002373332-Knowledge-Base">documentation</a>, and support <a href="https://support.count.ly/hc/en-us/sections/360007310512-SDKs">loads</a> of platforms.</p>
<p>They <a href="https://count.ly/pricing">offer</a> a paid Enterprise version or a free Community, self-hosted, option. I went with the Community edition.</p>
<p>The only significant missing feature (for me at least) in the Community version is no <a href="https://developer.apple.com/library/archive/technotes/tn2151/_index.html">symbolication</a> and de-obfuscation of crash traces.</p>
<p>Also, if you’re installing onto a server where you already have Nginx set up, you have to run the install scripts carefully.</p>
<h2 id="server-installation">Server Installation</h2>
<p>They have many install options, including a <a href="https://raw.githubusercontent.com/Countly/countly-server/master/bin/installer.sh">one-liner install script</a> that downloads the rest of the server and tries to install and configure your server. I did it manually.</p>
<p>Get the latest release and unzip:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">LATEST</span><span class="o">=</span><span class="si">$(</span>wget <span class="nt">-qO-</span> https://api.github.com/repos/countly/countly-server/releases/latest | <span class="nb">grep </span>browser_download_url | <span class="nb">head</span> <span class="nt">-n</span> 1 | <span class="nb">cut</span> <span class="nt">-d</span> <span class="s1">'"'</span> <span class="nt">-f</span> 4<span class="si">)</span> <span class="p">;</span>
<span class="nb">echo</span> <span class="s2">"Downloading from Github..."</span>
<span class="k">if</span> <span class="o">[[</span> <span class="s2">"</span><span class="nv">$LATEST</span><span class="s2">"</span> <span class="o">==</span> <span class="k">*</span>zip <span class="o">]]</span>
<span class="k">then
</span>wget <span class="nt">-nv</span> <span class="s2">"</span><span class="nv">$LATEST</span><span class="s2">"</span> <span class="nt">-O</span> ./countly.zip <span class="p">;</span>
unzip countly.zip
<span class="k">else
</span>wget <span class="nt">-nv</span> <span class="s2">"</span><span class="nv">$LATEST</span><span class="s2">"</span> <span class="nt">-O</span> ./countly.tar.gz <span class="p">;</span>
<span class="nb">tar </span>zxfv countly.tar.gz <span class="p">;</span>
<span class="k">fi</span>
</code></pre></div></div>
<p>Then run the installer script:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">DATE</span><span class="o">=</span><span class="si">$(</span><span class="nb">date</span> +%Y-%m-%d:%H:%M:%S<span class="si">)</span>
<span class="k">if</span> <span class="o">[[</span> <span class="o">!</span> <span class="nt">-z</span> <span class="s2">"</span><span class="nv">$APT_GET_CMD</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span>bash countly/bin/countly.install.sh 2>&1 | <span class="nb">tee</span> <span class="s2">"countly/log/countly-install-</span><span class="nv">$DATE</span><span class="s2">.log"</span>
<span class="k">elif</span> <span class="o">[[</span> <span class="o">!</span> <span class="nt">-z</span> <span class="s2">"</span><span class="nv">$YUM_CMD</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span>bash countly/bin/countly.install_rhel.sh 2>&1 | <span class="nb">tee</span> <span class="s2">"countly/log/countly-install-</span><span class="nv">$DATE</span><span class="s2">.log"</span>
<span class="k">else
</span><span class="nb">echo</span> <span class="s2">"error can't install Countly"</span>
<span class="nb">exit </span>1<span class="p">;</span>
<span class="k">fi</span>
</code></pre></div></div>
<p>Which, for Ubuntu, calls <a href="https://github.com/Countly/countly-server/blob/master/bin/countly.install_ubuntu.sh#L1">countly.install_ubuntu.sh</a></p>
<ul>
<li>Installs build dependencies</li>
<li>Installs <a href="https://nodejs.org/fr/blog/release/v10.0.0/">Node 10</a></li>
<li>Installs Nginx (I skipped this)</li>
<li>Installs <a href="http://supervisord.org/introduction.html">supervisor</a></li>
<li>Installs <a href="https://gruntjs.com/">grunt</a> & <a href="https://www.npmjs.com/">npm</a> modules</li>
<li>Installs <a href="https://www.mongodb.com/">MongoDB</a></li>
<li>Installs the Countly commands</li>
<li>Configures a default site for Nginx (I skipped this)</li>
<li>Installs various config files</li>
<li>Installs <a href="https://nghttp2.org/">Nghttp2</a></li>
<li>Installs Countly plugins</li>
<li>Builds Countly, then starts it.</li>
</ul>
<p>I had to do a bit of extra faffing to get Let’s Encrypt certificates and plug it into my already configured Nginx setup.</p>
<p>After that you need to:</p>
<ul>
<li><a href="https://support.count.ly/hc/en-us/articles/360037814871-Configuring-Countly">Configure</a> Countly</li>
<li><a href="https://support.count.ly/hc/en-us/articles/360037445752-Securing-MongoDB">Secure</a> MongoDB</li>
<li>Setup your <a href="https://support.count.ly/hc/en-us/articles/360037444232-Backing-up-Countly-server">backups</a></li>
<li>Learn the <a href="https://support.count.ly/hc/en-us/articles/360037444912-Countly-command-line">Countly command line</a></li>
</ul>
<p>Then login and create your admin user, create your apps, get the app keys and install the SDK for the <a href="https://support.count.ly/hc/en-us/articles/360037753511-iOS-watchOS-tvOS-macOS">apps</a> or <a href="https://support.count.ly/hc/en-us/articles/360037441932-Web-analytics-JavaScript-">Web site</a> you want to monitor.</p>
<h2 id="ios-sdk-installation">iOS SDK Installation</h2>
<p><a href="https://github.com/countly/countly-sdk-ios">Download</a> and install the SDK - just 32 files, all easy to comprehend.</p>
<p>Then instantiate Countly:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">CountlyConfig</span><span class="o">*</span> <span class="n">config</span> <span class="o">=</span> <span class="n">CountlyConfig</span><span class="p">.</span><span class="n">new</span><span class="p">;</span>
<span class="n">config</span><span class="p">.</span><span class="n">appKey</span> <span class="o">=</span> <span class="n">kCountlyAPIKey</span><span class="p">;</span>
<span class="n">config</span><span class="p">.</span><span class="n">host</span> <span class="o">=</span> <span class="n">kCountlyEndpoint</span><span class="p">;</span>
<span class="n">config</span><span class="p">.</span><span class="n">features</span> <span class="o">=</span> <span class="p">@[</span><span class="n">CLYCrashReporting</span><span class="p">,</span> <span class="n">CLYAutoViewTracking</span><span class="p">];</span>
<span class="n">config</span><span class="p">.</span><span class="n">starRatingDisableAskingForEachAppVersion</span> <span class="o">=</span> <span class="nb">YES</span><span class="p">;</span>
<span class="n">config</span><span class="p">.</span><span class="n">secretSalt</span> <span class="o">=</span> <span class="n">kCountlyAPISalt</span><span class="p">;</span>
<span class="n">config</span><span class="p">.</span><span class="n">alwaysUsePOST</span> <span class="o">=</span> <span class="nb">YES</span><span class="p">;</span>
<span class="p">[</span><span class="n">Countly</span><span class="p">.</span><span class="n">sharedInstance</span> <span class="nf">startWithConfig</span><span class="p">:</span><span class="n">config</span><span class="p">];</span>
<span class="p">[</span><span class="n">Countly</span><span class="p">.</span><span class="n">sharedInstance</span> <span class="nf">disableLocationInfo</span><span class="p">];</span>
</code></pre></div></div>
<p>That’s about it. There are <em>many</em> things you can configure on both the SDK and the server. Check the docs for that.</p>
<p>One note of warning, the SDK, by default, uses the <a href="https://developer.apple.com/documentation/adsupport/asidentifiermanager?language=objc">Advertising Identifier (IDFA)</a>, so you have to tell Apple why you’re using it, or disable it entirely. See the <a href="https://support.count.ly/hc/en-us/articles/360037753511-iOS-watchOS-tvOS-macOS#app-store-connect-idfa-warning">Countly docs</a>.</p>
<h2 id="web-analytics-installation">Web Analytics Installation</h2>
<p>You need to either add <code class="language-plaintext highlighter-rouge">sdk/web/countly.min.js</code> from your Countly server to your header code for the site you want to track, or grab the file and add it to your site’s build process. That’s what I did.</p>
<p>Added it to assets:</p>
<p><img data-src="/assets/images/posts/countly/assets.png" class="lazyload align-center blur-up" alt="assets" /></p>
<p>Created a setup html file:</p>
<p><a href="/assets/images/posts/countly/setup.png"><img data-src="/assets/images/posts/countly/setup.png" class="lazyload align-center blur-up" alt="setup" /></a></p>
<p>Then added it to my custom scripts <code class="language-plaintext highlighter-rouge">_include</code>:</p>
<p><a href="/assets/images/posts/countly/scripts.png"><img data-src="/assets/images/posts/countly/scripts.png" class="lazyload align-center blur-up" alt="scripts" /></a></p>
<p>So I now have some analytics trickling in:</p>
<p><a href="/assets/images/posts/countly/HKWarnings.png"><img data-src="/assets/images/posts/countly/HKWarnings.png" class="lazyload align-center blur-up" alt="HKWarnings" /></a></p>
<p><a href="/assets/images/posts/countly/stouty.png"><img data-src="/assets/images/posts/countly/stouty.png" class="lazyload align-center blur-up" alt="stouty" /></a></p>James StoutInstalling Countly server and SDK for iOS and Web analyticsFixing Nextcloud Conflicting Files2020-05-17T21:00:30+08:002020-05-17T21:00:30+08:00https://stouty.xyz/nextcloud/2020/05/17/nextcloud-conflicting-files<p>I made a bit of a mess with my Nextcloud sync yesterday. I have my Dropbox folder synced to my Mac Nextcloud folder, but not to the sync folder on my NAS. I thought I would save time and bandwidth by copying the folder from my Mac to my NAS and seeing if the sync could figure it out.</p>
<p>It didn’t.</p>
<p>I ended up with loads of conflicts. For example:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /Volumes/docker/nextcloud/media/Dropbox/Code/HKWarnings
<span class="nv">$ </span>find <span class="nb">.</span> <span class="nt">-name</span> <span class="s1">'*conflict*'</span>
./Classes/AboutViewControlleriPad <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.m
./Classes/AboutViewControlleriPad <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.h
./Classes/AboutViewController <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/Acknowledgements_iPhoneViewController <span class="o">(</span>conflicted copy 2020-04-28 103701<span class="o">)</span>.m
./Classes/Constants <span class="o">(</span>conflicted copy 2020-05-16 172756<span class="o">)</span>.h
./Classes/DisclaimerViewController2 <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/DetailViewController2 <span class="o">(</span>conflicted copy 2020-05-16 172756<span class="o">)</span>.m
./Classes/FCReachability <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/ForecastViewController <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/GetHistoricalWarnings <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/GADMasterViewController <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/HKWWarnings <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/HKWPhotosViewController <span class="o">(</span>conflicted copy 2020-05-14 193739<span class="o">)</span>.h
./Classes/HKWarningsWeatherImages <span class="o">(</span>conflicted copy 2020-05-11 083931<span class="o">)</span>.m
./Classes/HKWPhotosViewController <span class="o">(</span>conflicted copy 2020-05-14 193739<span class="o">)</span>.m
./Classes/HKWeatherWarningsTabsAppDelegate <span class="o">(</span>conflicted copy 2020-05-16 172756<span class="o">)</span>.h
./Classes/HKWeatherWarningsTabsAppDelegate <span class="o">(</span>conflicted copy 2020-05-16 172756<span class="o">)</span>.m
./Classes/HomeViewController <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.h
./Classes/HKWeatherWarningsiPadAppDelegate <span class="o">(</span>conflicted copy 2020-05-16 172756<span class="o">)</span>.m
./Classes/HKWeatherWarningsiPadAppDelegate <span class="o">(</span>conflicted copy 2020-05-16 172756<span class="o">)</span>.h
./Classes/HomeViewController <span class="o">(</span>conflicted copy 2020-05-05 135634<span class="o">)</span>.m
./Classes/InAppPurchaseViewController2 <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/InAppPurchaseManager <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/InAppPurchaseViewController_iPad <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/MYUIActivityItemProvider <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/MyUIActivity <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/NSDate+HKWUtilities <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.h
./Classes/NSMutableString+HKWUtilities <span class="o">(</span>conflicted copy 2020-05-08 172828<span class="o">)</span>.m
./Classes/NSDate+HKWUtilities <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/NSObject+JSON <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/NSString_HKWUtilities <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.h
./Classes/NSString_HKWUtilities <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/PopoverContentViewController <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/OptOutViewController_iPad <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/OptOutViewController_iPhone <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/PushSettingsViewController_iPad <span class="o">(</span>conflicted copy 2020-04-20 100727<span class="o">)</span>.m
./Classes/RadarViewController <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.h
./Classes/RadarViewController <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.m
./Classes/PushSettingsViewController_iPhone <span class="o">(</span>conflicted copy 2020-04-28 103701<span class="o">)</span>.m
./Classes/RadarViewController <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.xib
./Classes/SatViewController <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.h
./Classes/SatViewController <span class="o">(</span>conflicted copy 2020-05-03 151942<span class="o">)</span>.m
</code></pre></div></div>
<p>In total:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /Volumes/docker/nextcloud/media/Dropbox
<span class="nv">$ </span>find <span class="nb">.</span> <span class="nt">-name</span> <span class="s1">'*conflict*'</span> | <span class="nb">wc</span> <span class="nt">-l</span>
620
</code></pre></div></div>
<p>Right. I couldn’t fix them manually, or at least didn’t want to.</p>
<p>Script time.</p>
<p>The script below has been though a few iterations, to deal with spaces in filenames/directories, and files without an extension. It mostly works, but some files fail, and maybe some symlinks.</p>
<p>Now:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /Volumes/docker/nextcloud/media/Dropbox
<span class="nv">$ </span>find <span class="nb">.</span> <span class="nt">-name</span> <span class="s1">'*conflict*'</span> | <span class="nb">wc</span> <span class="nt">-l</span>
10
james@Jamess-iMac: /Volumes/docker/nextcloud/media/Dropbox
<span class="nv">$ </span>find <span class="nb">.</span> <span class="nt">-name</span> <span class="s1">'*conflict*'</span>
./Apps/Yojimbo/Database.bbyojimbostorage/Contents/Database/Database <span class="o">(</span>Jamess-iMac<span class="s1">'s conflicted copy 2018-07-07).sqlite
./Apps/Yojimbo/Database.bbyojimbostorage/Contents/Database/Database (Jamess-iMac'</span>s conflicted copy 2018-07-07<span class="o">)</span>.sqlite-shm
./Apps/Yojimbo/Database.bbyojimbostorage/Contents/Database/Database <span class="o">(</span>Jamess-iMac<span class="s1">'s conflicted copy 2018-07-07).sqlite-wal
./Apps/Yojimbo/State (Jamess-MacBook-Air'</span>s conflicted copy 2020-03-25<span class="o">)</span>.plist
./Apps/Yojimbo/State <span class="o">(</span>Jamess-iMac<span class="s1">'s conflicted copy 2020-01-24).plist
./Apps/Yojimbo/State (Jamess-iMac'</span>s conflicted copy 2020-03-25<span class="o">)</span>.plist
./Code/HKWarnings-server/1604/etc/monit/conf-available/mysql <span class="o">(</span>conflicted copy 2017-11-28 110223<span class="o">)</span>
./Code/HKWarnings-server/1604/etc/monit/conf-available/rsyslog <span class="o">(</span>conflicted copy 2017-11-28 110223<span class="o">)</span>
./Code/HKWarnings-server/nginx-config.io/sites-enabled/01_stouty.xyz <span class="o">(</span>conflicted copy 2019-10-11 110101<span class="o">)</span>.conf
./Code/HKWarnings-server/nginx-config.io/sites-enabled/02_hkwarnings.com <span class="o">(</span>conflicted copy 2019-10-11 110125<span class="o">)</span>.conf
</code></pre></div></div>
<p>The last 4 are symlinks. Not sure about the others.</p>
<p>Here’s the script:</p>
<script src="https://gist.github.com/jamesstout/f7cbe3def672f00a9bd6a7b008d7704f.js"></script>James StoutA bash script to correct conflicting files.Firebase Crashlytics - Are They Having a Laugh?2020-05-15T22:22:55+08:002020-05-15T22:22:55+08:00https://stouty.xyz/ios/2020/05/15/firebase-crashlytics<p>Crash reporting is an important part of app development, you don’t want your app to crash on users, so when it does it’s important to know when and why and be able to fix the issue.</p>
<p>I was a beta tester for <a href="https://en.wikipedia.org/wiki/Crashlytics">Crashlytics</a> back in 2012, exchanging emails with <a href="https://en.wikipedia.org/wiki/Wayne_Chang">Wayne Chang</a>, one of the founders:</p>
<p><a href="/assets/images/posts/firebase-crashlytics/wayne.png"><img data-src="/assets/images/posts/firebase-crashlytics/wayne.png" class="lazyload align-center blur-up" alt="wayne chang email" /></a></p>
<p>We exchanged feedback for a few months as they rolled out the SDK and the Mac app.</p>
<p>I like to think I was, in part, responsible for getting rid of the awful zipper thing they had on early versions of the crash report Web page:</p>
<p><img data-src="/assets/images/posts/firebase-crashlytics/zipper.png" class="lazyload align-center blur-up" alt="wayne chang email" /></p>
<p>I think <a href="https://xconomy.com/boston/2013/02/05/twitters-boston-acquisitions-crashlytics-tops-100m-bluefin-labs-close-behind/">Twitter bought them</a>, then in January 2017, <a href="https://fabric.io/blog/fabric-joins-google">Crashlytics and Fabric were acquired by Google</a>.</p>
<p>In September 2018 Google announced that <a href="https://venturebeat.com/2018/09/14/google-is-killing-fabric-in-mid-2019-pushes-developers-to-firebase/">Fabric will be deprecated</a> and developers should use Crashlytics via Firebase.</p>
<p>They’re a little late on their roadmap (by a year or so), but finally, this week:</p>
<p><a href="/assets/images/posts/firebase-crashlytics/fabric.png"><img data-src="/assets/images/posts/firebase-crashlytics/fabric.png" class="lazyload align-center blur-up" alt="fabruc shutdown" /></a></p>
<p>Developers have until November 15th 2020 for migrate to <a href="https://firebase.google.com/docs/crashlytics">Firebase Crashlytics</a>, or, another crash reporter.</p>
<p><img data-src="/assets/images/posts/firebase-crashlytics/fabric2.png" class="lazyload align-center blur-up" alt="fabric2" /></p>
<p>The current Fabric/Crashlytics SDK still works, you just have to go to the Firebase console to see the crash report. I currently have one outstanding crash in an old version, due to <a href="http://www.qnx.com/developers/docs/qnxcar2/index.jsp?topic=%2Fcom.qnx.doc.neutrino.prog%2Ftopic%2Fhat_ProblemsWithHeapCorruption.html">heap corruption</a>. Here’s an example of some of the data you can get:</p>
<figure class="half ">
<a href="/assets/images/posts/firebase-crashlytics/cr1.png" title="Example crash report">
<img src="/assets/images/posts/firebase-crashlytics/cr1-th.png" alt="Example crash report" />
</a>
<a href="/assets/images/posts/firebase-crashlytics/cr2.png" title="Example crash report">
<img src="/assets/images/posts/firebase-crashlytics/cr2-th.png" alt="Example crash report" />
</a>
<a href="/assets/images/posts/firebase-crashlytics/cr3.png" title="Example crash report">
<img src="/assets/images/posts/firebase-crashlytics/cr3-th.png" alt="Example crash report" />
</a>
<a href="/assets/images/posts/firebase-crashlytics/cr4.png" title="Example crash report">
<img src="/assets/images/posts/firebase-crashlytics/cr4-th.png" alt="Example crash report" />
</a>
</figure>
<p>Anyway, to the point of this post. The migration guide says to simply switch your Pods, add the plist and change the way you initiate the shared Crashlytics object:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Add the pod for Firebase Crashlytics</span>
<span class="n">pod</span> <span class="s1">'Firebase/Crashlytics'</span>
</code></pre></div></div>
<p>But check out what else it installs:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~/Projects/iPhone Apps/HKWarnings on fb[!<span class="nv">$]</span>
<span class="nv">$ </span>pod <span class="nb">install</span> <span class="nt">--no-repo-update</span>
Analyzing dependencies
Downloading dependencies
Installing Firebase <span class="o">(</span>6.24.0<span class="o">)</span>
Installing FirebaseAnalyticsInterop <span class="o">(</span>1.5.0<span class="o">)</span>
Installing FirebaseCore <span class="o">(</span>6.7.0<span class="o">)</span>
Installing FirebaseCoreDiagnostics <span class="o">(</span>1.3.0<span class="o">)</span>
Installing FirebaseCoreDiagnosticsInterop <span class="o">(</span>1.2.0<span class="o">)</span>
Installing FirebaseCrashlytics <span class="o">(</span>4.1.0<span class="o">)</span>
Installing FirebaseInstallations <span class="o">(</span>1.2.0<span class="o">)</span>
Installing GoogleDataTransport <span class="o">(</span>6.1.0<span class="o">)</span>
Installing GoogleDataTransportCCTSupport <span class="o">(</span>3.1.0<span class="o">)</span>
Installing GoogleUtilities 6.6.0
Removing Crashlytics
Removing Fabric
Generating Pods project
Integrating client project
Pod installation <span class="nb">complete</span><span class="o">!</span> There are 23 dependencies from the Podfile and 38 total pods installed.
</code></pre></div></div>
<p>Extra pods installed:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="pi">-</span> <span class="s">Firebase/CoreOnly (6.24.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">FirebaseCore (= 6.7.0)</span>
<span class="pi">-</span> <span class="s">Firebase/Crashlytics (6.24.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">Firebase/CoreOnly</span>
<span class="pi">-</span> <span class="s">FirebaseCrashlytics (~> 4.1.0)</span>
<span class="pi">-</span> <span class="s">FirebaseAnalyticsInterop (1.5.0)</span>
<span class="pi">-</span> <span class="s">FirebaseCore (6.7.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">FirebaseCoreDiagnostics (~> 1.3)</span>
<span class="pi">-</span> <span class="s">FirebaseCoreDiagnosticsInterop (~> 1.2)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/Environment (~> 6.5)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/Logger (~> 6.5)</span>
<span class="pi">-</span> <span class="s">FirebaseCoreDiagnostics (1.3.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">FirebaseCoreDiagnosticsInterop (~> 1.2)</span>
<span class="pi">-</span> <span class="s">GoogleDataTransportCCTSupport (~> 3.1)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/Environment (~> 6.5)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/Logger (~> 6.5)</span>
<span class="pi">-</span> <span class="s">nanopb (~> 1.30905.0)</span>
<span class="pi">-</span> <span class="s">FirebaseCoreDiagnosticsInterop (1.2.0)</span>
<span class="pi">-</span> <span class="s">FirebaseCrashlytics (4.1.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">FirebaseAnalyticsInterop (~> 1.2)</span>
<span class="pi">-</span> <span class="s">FirebaseCore (~> 6.6)</span>
<span class="pi">-</span> <span class="s">FirebaseInstallations (~> 1.1)</span>
<span class="pi">-</span> <span class="s">GoogleDataTransport (~> 6.1)</span>
<span class="pi">-</span> <span class="s">GoogleDataTransportCCTSupport (~> 3.1)</span>
<span class="pi">-</span> <span class="s">nanopb (~> 1.30905.0)</span>
<span class="pi">-</span> <span class="s">PromisesObjC (~> 1.2)</span>
<span class="pi">-</span> <span class="s">FirebaseInstallations (1.2.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">FirebaseCore (~> 6.6)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/Environment (~> 6.6)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/UserDefaults (~> 6.6)</span>
<span class="pi">-</span> <span class="s">PromisesObjC (~> 1.2)</span>
<span class="pi">-</span> <span class="s">GoogleDataTransport (6.1.0)</span>
<span class="pi">-</span> <span class="s">GoogleDataTransportCCTSupport (3.1.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">GoogleDataTransport (~> 6.1)</span>
<span class="pi">-</span> <span class="s">nanopb (~> 1.30905.0)</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/UserDefaults (6.6.0)</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">GoogleUtilities/Logger</span>
</code></pre></div></div>
<p>665 extra files, 3MB, Xcode build tasks go from 740 to 1600.</p>
<p>Build time goes from 43 to 70 seconds, 60% longer.</p>
<p><strong>What are all those files doing?!</strong></p>
<p>You can check out the full <a href="https://github.com/firebase/firebase-ios-sdk">Firebase platform on GitHub</a>.</p>
<p>Anyway, I’m not very keen, but I’ll give it a go. You can see how many iterations of the SDKs I’ve been through by the commented code:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">@try</span> <span class="p">{</span>
<span class="c1">//[Crashlytics startWithAPIKey:kCrashlyticsKey];</span>
<span class="c1">//[[Fabric sharedSDK] setDebug: FABRIC_DEBUG];</span>
<span class="c1">//[Fabric with:@[CrashlyticsKit]];</span>
<span class="c1">//[[Fabric with:@[[Crashlytics class]]] setDebug:FABRIC_DEBUG];</span>
<span class="p">[</span><span class="n">FIRApp</span> <span class="nf">configure</span><span class="p">];</span>
<span class="p">}</span>
<span class="k">@catch</span> <span class="p">(</span><span class="n">NSException</span> <span class="o">*</span><span class="n">exception</span><span class="p">)</span> <span class="p">{</span>
<span class="n">NSLog</span><span class="p">(</span><span class="s">@"[FIRApp configure]; Caught %@: %@"</span><span class="p">,</span> <span class="p">[</span><span class="n">exception</span> <span class="nf">name</span><span class="p">],</span> <span class="p">[</span><span class="n">exception</span> <span class="nf">reason</span><span class="p">]);</span>
<span class="n">startedCrashlytics</span> <span class="o">=</span> <span class="nb">NO</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Verbose logging from the SDK. It’s doing a lot, including downloading settings and logging each screen (even though it does nothing with it as I didn’t install Firebase Analytics).</p>
<script src="https://gist.github.com/jamesstout/444216d80bff511c96aa7927e8c73b55.js"></script>
<p>Build artifacts and app sizes:</p>
<figure class="half ">
<a href="/assets/images/posts/firebase-crashlytics/archive-no-firebase.png" title="archive-no-firebase">
<img src="/assets/images/posts/firebase-crashlytics/archive-no-firebase-th.png" alt="archive-no-firebase" />
</a>
<a href="/assets/images/posts/firebase-crashlytics/archive-with-firebase.png" title="archive-with-firebase">
<img src="/assets/images/posts/firebase-crashlytics/archive-with-firebase-th.png" alt="archive-with-firebase" />
</a>
<a href="/assets/images/posts/firebase-crashlytics/build-no-firebase.png" title="build-no-firebase">
<img src="/assets/images/posts/firebase-crashlytics/build-no-firebase-th.png" alt="build-no-firebase" />
</a>
<a href="/assets/images/posts/firebase-crashlytics/build-with-firebase.png" title="build-with-firebase">
<img src="/assets/images/posts/firebase-crashlytics/build-with-firebase-th.png" alt="build-with-firebase" />
</a>
</figure>
<p>1MB or so larger binary and dSYM.</p>
<p>There’s just too much going on that isn’t clear or documented. I’m going to have to look for alternatives.</p>
<ul>
<li><a href="https://www.bugsnag.com/platforms/ios-crash-reporting">Bugsnag</a> - free for up to 7500 events.</li>
<li><a href="https://count.ly/">Countly</a> - looks good, open source, free and self-hosted.</li>
<li><a href="https://sentry.io/">Sentry</a> - Uses Pod frameworks or <a href="https://github.com/Carthage/Carthage">Carthage</a>, and a <a href="https://github.com/getsentry/sentry-cocoa/releases">16MB framework</a>…🧐</li>
</ul>
<p>I think I’ll check out Countly first.</p>James StoutInvestigating the new Firebase Crashlytics SDK.A Jekyll Hook to Copy Pre-Built Site to Another Directory2020-05-11T23:42:03+08:002020-05-11T23:42:03+08:00https://stouty.xyz/webdev/2020/05/11/copy-site-jekyll-hook<p>This <a href="https://stouty.xyz/webdev/2020/04/28/git-deploy/">site</a> doesn’t build as a <a href="https://pages.github.com/">GitHub Page</a> as it uses <a href="https://pages.github.com/versions/">unsupported gems and plugins</a>. So when I push to <code class="language-plaintext highlighter-rouge">jamesstout.github.io.git</code> I get build failures.</p>
<p>My solution is to have two repositories, the source where I write and build the site, and a destination repo that points to the <code class="language-plaintext highlighter-rouge">jamesstout.github.io.git</code> <a href="https://github.com/jamesstout/jamesstout.github.io">repo</a>. The difference is, the destination repo just contains the files from the already built <code class="language-plaintext highlighter-rouge">_site</code> folder. This is what is pushed to the destination, and ultimately rendered at <a href="https://jamesstout.github.io/">jamesstout.github.io</a>.</p>
<p>Not wanting to do anything manually, I wrote a <a href="https://jekyllrb.com/docs/plugins/hooks/">Jekyll hook plugin</a> that copies the files from the <code class="language-plaintext highlighter-rouge">_site</code> folder to the destination repo, commits the changes and pushes to its remote, but only if there is a new commit in the source repo. No point copying, and pushing if you haven’t yet committed your changes. Here’s the plugin:</p>
<script src="https://gist.github.com/jamesstout/42201bbc498a4b91a9a1f884a17ced8c.js"></script>James StoutMy first attempt at a Jekyll plugin, it copies the `_site` folder somewhere else.A Quick Update on Zoom Code Signing2020-05-11T00:32:38+08:002020-05-11T00:32:38+08:00https://stouty.xyz/macos/2020/05/11/zoom-update<p>A quick update to my previous post on <a href="https://stouty.xyz/macos/2020/04/05/zoom/">Zoom security</a>.</p>
<p>Per <a href="https://twitter.com/scotteh/">@scotteh</a> and @<a href="https://twitter.com/patrickwardle/">patrickwardle</a>, you can still unsign the app and load any dylibs you like:
<!-- markdownlint-disable MD034 --></p>
<div class="jekyll-twitter-plugin"><blockquote class="twitter-tweet"><p lang="en" dir="ltr">Sure!<br />a) Unsign (via --remove-signature)<br />b) Load any dylibs 🤭<br /><br />e.g.<br />DYLD_INSERT_LIBRARIES=zoomzoom.dylib /Applications/zoomzoom.us .app/Contents/MacOS/zoom.us <br /><br />You'll have to (re)grant access to mic/webcam and notarization would be an issue if you (re)distribute though... <a href="https://t.co/RReUqvXF6S">pic.twitter.com/RReUqvXF6S</a></p>— patrick wardle (@patrickwardle) <a href="https://twitter.com/patrickwardle/status/1250684302049726464?ref_src=twsrc%5Etfw">April 16, 2020</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
</div>
<!-- markdownlint-enable MD034 -->
<p>Maybe I don’t understand <a href="https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html">code signing</a>, <a href="https://www.manpagez.com/man/8/spctl/">spctl</a> or <a href="https://en.wikipedia.org/wiki/Gatekeeper_(macOS)#Execution">Gatekeeper</a> well enough.</p>
<p>Gatekeeper is on, <code class="language-plaintext highlighter-rouge">spctl</code> says rejected, but Zoom still opens after removing the signature… 🧐</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~
<span class="nv">$ </span>spctl <span class="nt">--status</span> <span class="nt">-v</span>
assessments enabled
developer <span class="nb">id </span>enabled
james@Jamess-iMac: ~
<span class="nv">$ </span>spctl <span class="nt">--assess</span> <span class="nt">--verbose</span><span class="o">=</span>4 /Applications/zoom.us.app/
/Applications/zoom.us.app/: accepted
<span class="nb">source</span><span class="o">=</span>Developer ID
james@Jamess-iMac: ~
<span class="nv">$ </span><span class="nb">sudo </span>codesign <span class="nt">--remove-signature</span> /Applications/zoom.us.app/
james@Jamess-iMac: ~
<span class="nv">$ </span>spctl <span class="nt">--assess</span> <span class="nt">--verbose</span><span class="o">=</span>4 /Applications/zoom.us.app/
/Applications/zoom.us.app/: rejected
<span class="nb">source</span><span class="o">=</span>no usable signature
james@Jamess-iMac: ~
<span class="nv">$ </span>o /Applications/zoom.us.app/
james@Jamess-iMac: ~
<span class="nv">$ </span><span class="nb">echo</span> <span class="nv">$?</span>
0
</code></pre></div></div>James StoutA quick update on Zoom code signing.Compress-Upload-CocoaLumberjack - My First CocoaPod2020-05-09T23:09:06+08:002020-05-09T23:09:06+08:00https://stouty.xyz/ios/2020/05/09/compress-upload-cocoalumberjack<p>I just published my first CocoaPod: <a href="https://cocoapods.org/pods/Compress-Upload-CocoaLumberjack">Compress-Upload-CocoaLumberjack</a>:</p>
<blockquote>
<p>Remote logging via NSURLSession transfer to upload compressed CocoaLumberjack logs to an HTTP server.</p>
</blockquote>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~/Projects/Compress-Upload-CocoaLumberjack on master[?]
<span class="nv">$ </span>pod trunk me
- Name: James Stout
- Email: stoutyhk@gmail.com
- Since: May 7th, 20:14
- Pods:
- Compress-Upload-CocoaLumberjack
- Sessions:
- May 7th, 20:14 - September 14th, 21:44. IP: 203.218.233.190 Description: imac
james@Jamess-iMac: ~/Projects/Compress-Upload-CocoaLumberjack on master[?]
<span class="nv">$ </span>pod search Compress-Upload-CocoaLumberjack 2> /dev/null
-> Compress-Upload-CocoaLumberjack <span class="o">(</span>0.1.2<span class="o">)</span>
Remote logging via NSURLSession transfer to upload compressed CocoaLumberjack logs to an HTTP server.
pod <span class="s1">'Compress-Upload-CocoaLumberjack'</span>, <span class="s1">'~> 0.1.2'</span>
- Homepage: https://github.com/jamesstout/Compress-Upload-CocoaLumberjack
- Source: https://github.com/jamesstout/Compress-Upload-CocoaLumberjack.git
- Versions: 0.1.2, 0.1.1, 0.1.0 <span class="o">[</span>Specs repo]
</code></pre></div></div>
<p>The rationale for creating this Pod was that the <a href="https://github.com/CocoaLumberjack/CocoaLumberjack/tree/master/Demos/LogFileCompressor">demo log file compression project</a> in <a href="https://github.com/CocoaLumberjack/CocoaLumberjack">CocoaLumberjack</a> just zips archived files, and <a href="https://github.com/pushd/BackgroundUpload-CocoaLumberjack">BackgroundUpload-CocoaLumberjack</a> only uploads uncompressed log files. I wanted to compress, and then upload, to save space and bandwidth.</p>
<p><em>So first things first, thank you to <a href="https://github.com/pushd">pushd</a> and <a href="https://cocoalumberjack.github.io/">CocoaLumberjack</a> for their work, inspiration, examples and permissive licenses. This wouldn’t exist without them.</em></p>
<p>CocoaLumberjack is a fast & simple, yet powerful & flexible logging framework</p>
<blockquote>
<h2 id="lumberjack-is-fast--simple-yet-powerful--flexible">Lumberjack is Fast & Simple, yet Powerful & Flexible.</h2>
<p>It is similar in concept to other popular logging frameworks such as log4j, yet is designed specifically for Objective-C, and takes advantage of features such as multi-threading, grand central dispatch (if available), lockless atomic operations, and the dynamic nature of the Objective-C runtime.</p>
<h2 id="lumberjack-is-fast">Lumberjack is Fast</h2>
<p>In most cases it is an order of magnitude faster than NSLog.</p>
<h2 id="lumberjack-is-powerful">Lumberjack is Powerful</h2>
<p>One log statement can be sent to multiple loggers, meaning you can log to a file and the console simultaneously. Want more? Create your own loggers (it’s easy) and send your log statements over the network. Or to a database or distributed file system. The sky is the limit.</p>
</blockquote>
<p>Check out the <a href="https://github.com/CocoaLumberjack/CocoaLumberjack/blob/master/Documentation/Performance.md">benchmarks</a>.</p>
<p>The default philosophy for asynchronous logging is very simple:</p>
<ul>
<li>Log messages with errors should be executed synchronously.</li>
<li>All other log messages, such as debug output, are executed asynchronously.</li>
</ul>
<p>Getting started is simple. This will add a pair of “loggers” and your log statements will be sent to the Console.app and the Xcode console (just like a normal <code class="language-plaintext highlighter-rouge">NSLog</code>).</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#define LOG_LEVEL_DEF ddLogLevel
#import <CocoaLumberjack/CocoaLumberjack.h>
</span>
<span class="p">[</span><span class="n">DDLog</span> <span class="nf">addLogger</span><span class="p">:[</span><span class="n">DDOSLogger</span> <span class="nf">sharedInstance</span><span class="p">]];</span>
</code></pre></div></div>
<p>To add another logger, one that writes to a file:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">DDFileLogger</span> <span class="o">*</span><span class="n">fileLogger</span> <span class="o">=</span> <span class="p">[[</span><span class="n">DDFileLogger</span> <span class="nf">alloc</span><span class="p">]</span> <span class="nf">init</span><span class="p">];</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">rollingFrequency</span> <span class="o">=</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">24</span><span class="p">;</span> <span class="c1">// 24 hour rolling</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">logFileManager</span><span class="p">.</span><span class="n">maximumNumberOfLogFiles</span> <span class="o">=</span> <span class="mi">7</span><span class="p">;</span>
<span class="p">[</span><span class="n">DDLog</span> <span class="nf">addLogger</span><span class="p">:</span><span class="n">fileLogger</span><span class="p">];</span>
</code></pre></div></div>
<p>That’s a fairly standard log file setup, with one file per day, for a week. After a week, the oldest file is deleted.</p>
<p>Then to log, just convert your <code class="language-plaintext highlighter-rouge">NSLog</code> calls:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Convert from this:</span>
<span class="n">NSLog</span><span class="p">(</span><span class="s">@"Broken sprocket detected!"</span><span class="p">);</span>
<span class="n">NSLog</span><span class="p">(</span><span class="s">@"User selected file:%@ withSize:%u"</span><span class="p">,</span> <span class="n">filePath</span><span class="p">,</span> <span class="n">fileSize</span><span class="p">);</span>
<span class="c1">// To this:</span>
<span class="n">DDLogError</span><span class="p">(</span><span class="s">@"Broken sprocket detected!"</span><span class="p">);</span>
<span class="n">DDLogVerbose</span><span class="p">(</span><span class="s">@"User selected file:%@ withSize:%u"</span><span class="p">,</span> <span class="n">filePath</span><span class="p">,</span> <span class="n">fileSize</span><span class="p">);</span>
</code></pre></div></div>
<p>Those logs will be set to the console, and to the log file.</p>
<p>If you make the <code class="language-plaintext highlighter-rouge">logFileManager</code> a <code class="language-plaintext highlighter-rouge">CompressingLogFileManager</code>, per the LogFileCompressor demo project mentioned above, your setup looks like this:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">CompressingLogFileManager</span> <span class="o">*</span><span class="n">logFileManager</span> <span class="o">=</span> <span class="p">[[</span><span class="n">CompressingLogFileManager</span> <span class="nf">alloc</span><span class="p">]</span> <span class="nf">init</span><span class="p">];</span>
<span class="n">fileLogger</span> <span class="o">=</span> <span class="p">[[</span><span class="n">DDFileLogger</span> <span class="nf">alloc</span><span class="p">]</span> <span class="nf">initWithLogFileManager</span><span class="p">:</span><span class="n">logFileManager</span><span class="p">];</span>
<span class="c1">// 1kB and 1 min, just to demo quick rolling/compressing</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">maximumFileSize</span> <span class="o">=</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1</span><span class="p">;</span> <span class="c1">// 1 KB</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">rollingFrequency</span> <span class="o">=</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">1</span><span class="p">;</span> <span class="c1">// 1 Minute</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">logFileManager</span><span class="p">.</span><span class="n">maximumNumberOfLogFiles</span> <span class="o">=</span> <span class="mi">4</span><span class="p">;</span>
<span class="p">[</span><span class="n">DDLog</span> <span class="nf">addLogger</span><span class="p">:[</span><span class="n">DDOSLogger</span> <span class="nf">sharedInstance</span><span class="p">]];</span>
<span class="p">[</span><span class="n">DDLog</span> <span class="nf">addLogger</span><span class="p">:</span><span class="n">fileLogger</span><span class="p">];</span>
</code></pre></div></div>
<p>With this setup, your log file is ‘archived’ and then compressed to a <code class="language-plaintext highlighter-rouge">.gz</code> file. After a while, the log folder looks like this:</p>
<p><a href="/assets/images/posts/compress-upload-cocoalumberjack/log-folder.png"><img data-src="/assets/images/posts/compress-upload-cocoalumberjack/log-folder.png" class="lazyload align-center blur-up" alt="log-folder" /></a></p>
<p>Seems the <code class="language-plaintext highlighter-rouge">.gz</code> files are excluded from the deletion based on <code class="language-plaintext highlighter-rouge">maximumNumberOfLogFiles</code>, so the folder would just fill up, I think.</p>
<p>I deal with this in Compress-Upload-CocoaLumberjack. To set up Compress-Upload-CocoaLumberjack, you need an HTTP(S) server with an endpoint that accepts the log files, then instantiate the logger with your URL:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// change the URL to your domain/endpoint</span>
<span class="n">NSMutableURLRequest</span> <span class="o">*</span><span class="n">request</span> <span class="o">=</span> <span class="p">[</span><span class="n">NSMutableURLRequest</span>
<span class="nl">requestWithURL:</span><span class="p">[</span><span class="n">NSURL</span> <span class="nf">URLWithString</span><span class="p">:</span><span class="s">@"http://localhost:3000/log"</span><span class="p">]];</span>
<span class="p">[</span><span class="n">request</span> <span class="nf">setHTTPMethod</span><span class="p">:</span><span class="s">@"POST"</span><span class="p">];</span>
<span class="n">logFileManager</span> <span class="o">=</span> <span class="p">[[</span><span class="n">CompressingAndUploadingLogFileManager</span> <span class="nf">alloc</span><span class="p">]</span> <span class="nf">initWithUploadRequest</span><span class="p">:</span><span class="n">request</span><span class="p">];</span>
<span class="n">fileLogger</span> <span class="o">=</span> <span class="p">[[</span><span class="n">DDFileLogger</span> <span class="nf">alloc</span><span class="p">]</span> <span class="nf">initWithLogFileManager</span><span class="p">:</span><span class="n">logFileManager</span><span class="p">];</span>
<span class="c1">// set to 1 min and 1kB so they roll quickly</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">maximumFileSize</span> <span class="o">=</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">100</span><span class="p">;</span> <span class="c1">// 1 KB</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">rollingFrequency</span> <span class="o">=</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">1</span><span class="p">;</span> <span class="c1">// 1 Minute</span>
<span class="n">fileLogger</span><span class="p">.</span><span class="n">logFileManager</span><span class="p">.</span><span class="n">maximumNumberOfLogFiles</span> <span class="o">=</span> <span class="mi">4</span><span class="p">;</span>
<span class="p">[</span><span class="n">DDLog</span> <span class="nf">addLogger</span><span class="p">:[</span><span class="n">DDOSLogger</span> <span class="nf">sharedInstance</span><span class="p">]];</span>
<span class="p">[</span><span class="n">DDLog</span> <span class="nf">addLogger</span><span class="p">:</span><span class="n">fileLogger</span><span class="p">];</span>
</code></pre></div></div>
<p>The compression behaviour is the same as in LogFileCompressor, except in <code class="language-plaintext highlighter-rouge">compressionDidSucceed</code></p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#pragma mark - compressionDidSucceed
</span><span class="k">-</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="nf">compressionDidSucceed</span><span class="p">:(</span><span class="n">DDLogFileInfo</span> <span class="o">*</span><span class="p">)</span><span class="nv">logFile</span>
<span class="p">{</span>
<span class="n">NSLogVerbose</span><span class="p">(</span><span class="s">@"CompressingAndUploadingLogFileManager: compressionDidSucceed: %@"</span><span class="p">,</span> <span class="n">logFile</span><span class="p">.</span><span class="n">fileName</span><span class="p">);</span>
<span class="c1">// at this point we want to upload logFile, if the dontUpload flag isn't set</span>
<span class="k">if</span><span class="p">(</span><span class="n">doUpload</span> <span class="o">==</span> <span class="nb">YES</span><span class="p">){</span>
<span class="p">[</span><span class="n">self</span> <span class="nf">uploadArchivedFile</span><span class="p">:</span><span class="n">logFile</span><span class="p">];</span>
<span class="p">}</span>
<span class="n">self</span><span class="p">.</span><span class="n">isCompressing</span> <span class="o">=</span> <span class="nb">NO</span><span class="p">;</span>
<span class="p">[</span><span class="n">self</span> <span class="nf">compressNextLogFile</span><span class="p">];</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">uploadArchivedFile</code> eventually calls:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">-</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="nf">uploadLogFile</span><span class="p">:(</span><span class="n">NSString</span> <span class="o">*</span><span class="p">)</span><span class="nv">logFilePath</span>
<span class="p">{</span>
<span class="n">NSMutableURLRequest</span> <span class="o">*</span><span class="n">request</span> <span class="o">=</span> <span class="p">[</span><span class="n">self</span><span class="p">.</span><span class="n">uploadRequest</span> <span class="nf">mutableCopy</span><span class="p">];</span>
<span class="p">[</span><span class="n">request</span> <span class="nf">setValue</span><span class="p">:[</span><span class="n">logFilePath</span> <span class="nf">stringByAddingPercentEncodingWithAllowedCharacters</span><span class="p">:[</span><span class="n">NSCharacterSet</span> <span class="nf">URLHostAllowedCharacterSet</span><span class="p">]]</span> <span class="nf">forHTTPHeaderField</span><span class="p">:</span><span class="s">@"X-BackgroundUpload-File"</span><span class="p">];</span>
<span class="c1">// added extra header to identify upload</span>
<span class="p">[</span><span class="n">request</span> <span class="nf">setValue</span><span class="p">:[</span><span class="n">UIDevice</span><span class="p">.</span><span class="n">currentDevice</span><span class="p">.</span><span class="n">identifierForVendor</span><span class="p">.</span><span class="n">UUIDString</span> <span class="nf">stringByAddingPercentEncodingWithAllowedCharacters</span><span class="p">:[</span><span class="n">NSCharacterSet</span> <span class="nf">URLHostAllowedCharacterSet</span><span class="p">]]</span> <span class="nf">forHTTPHeaderField</span><span class="p">:</span><span class="s">@"X-BackgroundUpload-File-UUID"</span><span class="p">];</span>
<span class="n">NSURLSessionTask</span> <span class="o">*</span><span class="n">task</span> <span class="o">=</span> <span class="p">[</span><span class="n">self</span><span class="p">.</span><span class="n">session</span> <span class="nf">uploadTaskWithRequest</span><span class="p">:</span><span class="n">request</span> <span class="nf">fromFile</span><span class="p">:[</span><span class="n">NSURL</span> <span class="nf">fileURLWithPath</span><span class="p">:</span><span class="n">logFilePath</span><span class="p">]];</span>
<span class="n">task</span><span class="p">.</span><span class="n">taskDescription</span> <span class="o">=</span> <span class="n">logFilePath</span><span class="p">;</span>
<span class="p">[</span><span class="n">task</span> <span class="nf">resume</span><span class="p">];</span>
<span class="k">if</span> <span class="p">([</span><span class="n">self</span><span class="p">.</span><span class="n">delegate</span> <span class="nf">respondsToSelector</span><span class="p">:</span><span class="k">@selector</span><span class="p">(</span><span class="nf">attemptingUploadForFilePath</span><span class="p">:)])</span> <span class="p">{</span>
<span class="p">[</span><span class="n">self</span><span class="p">.</span><span class="n">delegate</span> <span class="nf">attemptingUploadForFilePath</span><span class="p">:</span><span class="n">logFilePath</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">X-BackgroundUpload-File</code> contains the filename. I added <code class="language-plaintext highlighter-rouge">X-BackgroundUpload-File-UUID</code> to anonymously identify each device so I could separate the log files on the server-side, and do a few security checks.</p>
<p>I’m not going to publish the server code, as I’m pretty sure it has some vulnerabilities! Something like this, in PHP:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="k">function</span> <span class="n">hhud</span><span class="p">(</span><span class="nv">$var</span><span class="p">,</span> <span class="nv">$index</span><span class="p">){</span>
<span class="k">return</span> <span class="k">isset</span><span class="p">(</span><span class="nv">$var</span><span class="p">[</span><span class="nv">$index</span><span class="p">])</span> <span class="o">?</span> <span class="nb">rawurldecode</span><span class="p">(</span><span class="nv">$var</span><span class="p">[</span><span class="nv">$index</span><span class="p">])</span> <span class="o">:</span> <span class="s1">''</span><span class="p">;</span>
<span class="p">}</span>
<span class="nv">$request_body</span> <span class="o">=</span> <span class="o">@</span><span class="nb">file_get_contents</span><span class="p">(</span><span class="s1">'php://input'</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="nv">$request_body</span> <span class="o">!=</span> <span class="kc">null</span> <span class="p">){</span>
<span class="c1">// do some checks and validation on input</span>
<span class="nv">$fullFileName</span> <span class="o">=</span> <span class="nf">hhud</span><span class="p">(</span><span class="nv">$_SERVER</span><span class="p">,</span> <span class="s2">"HTTP_X_BACKGROUNDUPLOAD_FILE"</span><span class="p">);</span>
<span class="c1">// File path to save file</span>
<span class="nv">$filePath</span> <span class="o">=</span> <span class="s1">'xxxxx/'</span><span class="p">;</span>
<span class="nv">$fileName</span> <span class="o">=</span> <span class="s1">''</span><span class="p">;</span>
<span class="nv">$fileDir</span> <span class="o">=</span> <span class="s1">'xxxxx/'</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="nv">$fullFileName</span> <span class="o">!=</span> <span class="s1">''</span><span class="p">){</span>
<span class="nv">$fileName</span> <span class="o">=</span> <span class="nb">basename</span><span class="p">(</span><span class="nv">$fullFileName</span><span class="p">);</span>
<span class="nv">$fileName</span> <span class="o">=</span> <span class="nb">preg_replace</span><span class="p">(</span><span class="s1">'/\s+/'</span><span class="p">,</span> <span class="s1">'-'</span><span class="p">,</span> <span class="nv">$fileName</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$filePath</span> <span class="o">=</span> <span class="nv">$filePath</span> <span class="mf">.</span> <span class="nv">$fileDir</span><span class="p">;</span>
<span class="nv">$file</span> <span class="o">=</span> <span class="nv">$filePath</span> <span class="mf">.</span> <span class="s2">"/"</span> <span class="mf">.</span> <span class="nv">$fileName</span><span class="p">;</span>
<span class="c1">// Write the request body to file</span>
<span class="k">if</span><span class="p">(</span><span class="nb">file_put_contents</span><span class="p">(</span><span class="nv">$file</span><span class="p">,</span> <span class="nv">$request_body</span><span class="p">)</span> <span class="o">===</span> <span class="kc">FALSE</span><span class="p">){</span>
<span class="nv">$serverDeets</span> <span class="o">=</span> <span class="nb">var_export</span><span class="p">(</span><span class="nv">$_SERVER</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span> <span class="s2">"Failed to write to file:"</span> <span class="mf">.</span> <span class="nv">$file</span> <span class="mf">.</span> <span class="s2">"</span><span class="se">\n\n\$</span><span class="s2">_SERVER was:</span><span class="se">\n</span><span class="s2"> </span><span class="nv">$serverDeets</span><span class="se">\n</span><span class="s2"> "</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">else</span><span class="p">{</span>
<span class="nb">http_response_code</span><span class="p">(</span><span class="mi">400</span><span class="p">);</span>
<span class="nb">http_response_code</span><span class="p">();</span>
<span class="k">exit</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>To deal with old compressed files in the logs directory, I added <code class="language-plaintext highlighter-rouge">uploadOldCompressedLogFiles</code> to the <code class="language-plaintext highlighter-rouge">CompressingAndUploadingLogFileManager init</code>:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">-</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="n">uploadOldCompressedLogFiles</span><span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">doUpload</span> <span class="o">==</span> <span class="nb">NO</span><span class="p">){</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">NSArray</span> <span class="o">*</span><span class="n">unsortedLogFileInfos</span> <span class="o">=</span> <span class="p">[</span><span class="n">self</span> <span class="nf">unsortedLogFileInfosGZ</span><span class="p">];</span>
<span class="k">for</span> <span class="p">(</span><span class="n">DDLogFileInfo</span> <span class="o">*</span><span class="n">fileInfo</span> <span class="k">in</span> <span class="n">unsortedLogFileInfos</span><span class="p">)</span> <span class="p">{</span>
<span class="n">NSLogVerbose</span><span class="p">(</span><span class="s">@"CompressingAndUploadingLogFileManager: fileInfo: %@"</span><span class="p">,</span> <span class="n">fileInfo</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">fileInfo</span><span class="p">.</span><span class="n">isArchived</span><span class="p">)</span> <span class="p">{</span>
<span class="n">NSLogVerbose</span><span class="p">(</span><span class="s">@"is Archived, uploading..."</span><span class="p">);</span>
<span class="p">[</span><span class="n">self</span> <span class="nf">uploadArchivedFile</span><span class="p">:</span><span class="n">fileInfo</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>You might have spotted that there’s a property on the LogFileManager that you can set to prevent uploads (it defaults to <code class="language-plaintext highlighter-rouge">YES</code>). To stop uploading, simply set to <code class="language-plaintext highlighter-rouge">NO</code>.</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">logFileManager</span><span class="p">.</span><span class="n">doUpload</span> <span class="o">=</span> <span class="nb">NO</span><span class="p">;</span>
</code></pre></div></div>
<p>In my <a href="https://apps.apple.com/us/app/hkwarnings/id370901118#?platform=ipad">HKWarnings app</a>, I can set this flag from my server, enabling me to remotely switch log file uploading on and off.</p>
<p>On the server, I end up with folders full of gzipped log files:</p>
<figure class="half ">
<a href="/assets/images/posts/compress-upload-cocoalumberjack/logs.png" title="Log files">
<img src="/assets/images/posts/compress-upload-cocoalumberjack/logs-th.png" alt="Log files" />
</a>
<a href="/assets/images/posts/compress-upload-cocoalumberjack/tree.png" title="Log tree">
<img src="/assets/images/posts/compress-upload-cocoalumberjack/tree-th.png" alt="Log tree" />
</a>
</figure>
<p>I take a copy, unzip and <code class="language-plaintext highlighter-rouge">grep</code> the logs for insights. For example, this lists debug lines and orders them by the number of times they occur:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>egrep <span class="s1">'\[\D\]'</span> <span class="k">*</span>.log | <span class="nb">grep</span> <span class="nt">-Po</span> <span class="s1">' \[.*'</span> | <span class="nb">sed</span> <span class="s1">'s/^ *//g'</span> | <span class="nb">sort</span> | <span class="nb">uniq</span> <span class="nt">-c</span> | <span class="nb">sort</span> <span class="nt">-n</span>
</code></pre></div></div>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1182 [D] [Warning -[Warning isItNight]][Line 1619] in isItNight
1343 [D] [WarningViewController3 -[WarningViewController3 showLabel]][Line 2152] have not purchased ads
1361 [D] [NSMutableString+HKWUtilities -[NSMutableString(HKWUtilities) hkw_base64EncodedURLQueryString:]][Line 14] encQueryString is []
1377 [D] [WarningViewController3 -[WarningViewController3 showLabel]][Line 2136] In showLabel
</code></pre></div></div>
<p>So I can see I should make sure these are performant:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">isItNight</code></li>
<li><code class="language-plaintext highlighter-rouge">hkw_base64EncodedURLQueryString</code></li>
</ul>
<p>In fact I’ve already released a version where <code class="language-plaintext highlighter-rouge">hkw_base64EncodedURLQueryString</code> is 66% faster and <code class="language-plaintext highlighter-rouge">isItNight</code> is ~30% faster:</p>
<figure class="half ">
<a href="/assets/images/posts/compress-upload-cocoalumberjack/isitnight.png" title="Is it night test">
<img src="/assets/images/posts/compress-upload-cocoalumberjack/isitnight-th.png" alt="Is it night test" />
</a>
<a href="/assets/images/posts/compress-upload-cocoalumberjack/isitnight2.png" title="Is it night 2 tests">
<img src="/assets/images/posts/compress-upload-cocoalumberjack/isitnight2-th.png" alt="Is it night 2 tests" />
</a>
</figure>
<p>Final test to check the output of the new version matches the old version:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">-</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="n">testisItNight_isItNight2</span> <span class="p">{</span>
<span class="n">Warning</span> <span class="o">*</span><span class="n">warning</span> <span class="o">=</span> <span class="p">[</span><span class="n">Warning</span> <span class="nf">alloc</span><span class="p">];</span>
<span class="n">BOOL</span> <span class="n">isitNight2</span> <span class="o">=</span> <span class="p">[</span><span class="n">warning</span> <span class="nf">isItNight2</span><span class="p">];</span>
<span class="n">BOOL</span> <span class="n">isitNight</span> <span class="o">=</span> <span class="p">[</span><span class="n">warning</span> <span class="nf">isItNight</span><span class="p">];</span>
<span class="n">expect</span><span class="p">(</span><span class="n">isitNight2</span><span class="p">).</span><span class="n">to</span><span class="p">.</span><span class="n">equal</span><span class="p">(</span><span class="n">isitNight</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Anyway, here it is in action (click through to view the video):</p>
<p><a href="https://www.dropbox.com/s/o0bvlqlo358tgc6/uploadingLogFiles-short.mp4?dl=0"><img data-src="/assets/images/posts/compress-upload-cocoalumberjack/uploadingLogFiles.png" src="/assets/images/posts/compress-upload-cocoalumberjack/uploadingLogFiles-lq.png" class="lazyload align-center blur-up" alt="Git files changes screenshot" /></a></p>James StoutMy first CocoaPod - a mashup of CompressingLogFileManager from CocoaLumberjack and BackgroundUpload-CocoaLumberjack.Nextcloud Sync Client for Docker2020-05-08T22:27:27+08:002020-05-08T22:27:27+08:00https://stouty.xyz/nextcloud/2020/05/08/nextcloud-sync<p>I’ve found the ‘native’ macOS Nextcloud client to be a bit flakey. It’s all <a href="https://www.qt.io/">Qt</a> and <a href="https://github.com/nextcloud/desktop/search?l=c%2B%2B">C++</a> and times out all the time.</p>
<p>I’m trying to replace Dropbox on my Mac, as they get up to all sorts of <a href="https://applehelpwriter.com/2016/07/28/revealing-dropboxs-dirty-little-security-hack/">scummy</a> <a href="https://applehelpwriter.com/2016/08/29/discovering-how-dropbox-hacks-your-mac/">behaviour</a>, but it’s a little harder than expected.</p>
<p>I mentioned in an earlier post that I <a href="https://stouty.xyz/unix/2020/04/16/dupes/">sync my photos to my NAS</a>, then sync to Nextcloud via their <a href="https://docs.nextcloud.com/desktop/2.6/advancedusage.html#nextcloud-command-line-client">command line client</a>, running in a <a href="https://www.docker.com/">Docker</a> <a href="https://www.docker.com/resources/what-container">container</a>. I run all my containers on my Synology NAS using their <a href="https://www.synology.com/en-global/dsm/feature/docker">Docker package</a>:</p>
<figure class="third ">
<a href="/assets/images/posts/nextcloud-sync/containers.png" title="Docker containers">
<img src="/assets/images/posts/nextcloud-sync/containers-th.png" alt="Docker containers" />
</a>
<a href="/assets/images/posts/nextcloud-sync/containers2.png" title="Docker containers">
<img src="/assets/images/posts/nextcloud-sync/containers2-th.png" alt="Docker containers" />
</a>
<a href="/assets/images/posts/nextcloud-sync/images.png" title="Docker images">
<img src="/assets/images/posts/nextcloud-sync/images-th.png" alt="Docker images" />
</a>
</figure>
<p>In the last screenshot there you can see my image: <a href="https://registry.hub.docker.com/r/stoutyhk/nextcloud-client/">stoutyhk/nextcloud-client</a>. It’s based on the image from <a href="https://github.com/juanitomint/nextcloud-client-docker">juanitomint</a>, with some <a href="https://github.com/jamesstout/nextcloud-client-docker/commit/c6184018dcc46289cbbb59060c63870814710e59">small tweaks</a> to add functionality to exclude certain folders and to set folders for selective sync. I also slightly tweaked how the arguments to the client are constructed and exposed the <code class="language-plaintext highlighter-rouge">media</code> and <code class="language-plaintext highlighter-rouge">config</code> folders.</p>
<figure class="half ">
<a href="/assets/images/posts/nextcloud-sync/Dockerfile.png" title="Dockerfile">
<img src="/assets/images/posts/nextcloud-sync/Dockerfile-th.png" alt="Dockerfile" />
</a>
<a href="/assets/images/posts/nextcloud-sync/run.png" title="run.sh">
<img src="/assets/images/posts/nextcloud-sync/run-th.png" alt="run.sh" />
</a>
</figure>
<p>The config folder can contain files named:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">exclude</code> - this contains files/folders/patterns that you don’t want to sync</li>
<li><code class="language-plaintext highlighter-rouge">unsyncedfolders</code> - contains the list of un-synced remote folders (selective sync)</li>
</ul>
<p>For example, I want to exclude syncing the Synology-generated metadata <a href="https://www.reddit.com/r/synology/comments/an4a4l/modern_solution_to_stopping_the_generation_of/">@eaDir</a> directories to my server:</p>
<p>So I have a config directory containing an <code class="language-plaintext highlighter-rouge">exclude</code> file:</p>
<p><a href="/assets/images/posts/nextcloud-sync/config.png"><img data-src="/assets/images/posts/nextcloud-sync/config.png" class="lazyload align-center blur-up" alt="config folder screenshot" /></a></p>
<p>Containing:</p>
<p><a href="/assets/images/posts/nextcloud-sync/exclude.png"><img data-src="/assets/images/posts/nextcloud-sync/exclude.png" class="lazyload align-center blur-up" alt="exclude file screenshot" /></a></p>
<p>Additionally, on my server there are folders that I do not want to sync down to my NAS. So those folders go in the <code class="language-plaintext highlighter-rouge">unsyncedfolders</code> file:</p>
<p><a href="/assets/images/posts/nextcloud-sync/unsyncedfolders.png"><img data-src="/assets/images/posts/nextcloud-sync/unsyncedfolders.png" class="lazyload align-center blur-up" alt="unsyncedfolders file screenshot" /></a></p>
<p>So you download the image, launch and set the <a href="https://docs.docker.com/storage/volumes/">volumes</a> and environment variables, and start it up:</p>
<p><a href="/assets/images/posts/nextcloud-sync/nc-docker-settings.png"><img data-src="/assets/images/posts/nextcloud-sync/nc-docker-settings.png" class="lazyload align-center blur-up" alt="nc-docker-settings screenshot" /></a></p>
<p>From the command line this would be something like:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-it</span> <span class="nt">--rm</span> <span class="se">\</span>
<span class="nt">-v</span> some_named_volume:/media/nextcloud <span class="se">\</span>
<span class="nt">-v</span> some_config_volume:/config <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">NC_USER</span><span class="o">=</span><span class="nv">$username</span> <span class="nt">-e</span> <span class="nv">NC_PASS</span><span class="o">=</span><span class="nv">$password</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">NC_URL</span><span class="o">=</span><span class="nv">$server_url</span><span class="se">\</span>
stoutyhk/nextcloud-client
</code></pre></div></div>
<p>The Dockerfile CMD is run.sh, a script that parses the arguments to construct the nextcloudcmd command. For my setup it’s this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s2">"/bin/su -s /bin/ash </span><span class="nv">$USER</span><span class="s2"> -c 'nextcloudcmd --exclude /config/exclude </span><span class="se">\</span><span class="s2">
--unsyncedfolders /config/unsyncedfolders --max-sync-retries 6 </span><span class="se">\</span><span class="s2">
--non-interactive -u </span><span class="nv">$NC_USER</span><span class="s2"> -p </span><span class="nv">$NC_PASS</span><span class="s2"> /media/nextcloud </span><span class="nv">$NC_URL</span><span class="s2">'"</span>
</code></pre></div></div>
<p>This simply runs in a <a href="https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_09_02.html">while true loop</a>, sleeping for 500s after it syncs and then checking if it needs to sync again after 500s.</p>
<p>It’s very stable, hasn’t crashed or timed out yet. I highly recommend this method vs the native macOS client.</p>
<p>Going full No Dropbox would mean syncing further folders and mounting them on my Mac. That’s up next.</p>
<p>BTW, building and tweaking this is very simple. The <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile</a> describes what to do:</p>
<p>Build image based on <a href="https://www.alpinelinux.org/">Alpine</a> (a tiny, 2.7MB, version of Linux):</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>FROM alpine:latest
</code></pre></div></div>
<p>Set some ARGS and ENVS:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ARG USER=ncsync
ARG GROUP=users
ARG USER_UID=1000
ARG USER_GID=100
ENV USER=$USER \
GROUP=$GROUP \
USER_UID=$USER_UID \
USER_GID=$USER_GID \
NC_USER=username \
NC_PASS=password \
NC_INTERVAL=500 \
NC_URL="" \
NC_TRUST_CERT=false \
NC_SILENT=false \
NC_EXIT=false \
NC_HIDDEN=false \
NC_MAX_SYNC_RETRIES=3
</code></pre></div></div>
<p>Create user/group:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN adduser -G $GROUP -D -u $USER_UID $USER
</code></pre></div></div>
<p>Update repositories and install nextcloud-client:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN apk update && apk add nextcloud-client && rm -rf /etc/apk/cache
</code></pre></div></div>
<p>Add run script and expose volumes:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ADD run.sh /usr/bin/run.sh
VOLUME [ "/media/nextcloud" ]
VOLUME [ "/config" ]
</code></pre></div></div>
<p>Finally, execute the command:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CMD /usr/bin/run.sh
</code></pre></div></div>
<p>The build command to build the image is:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker build <span class="se">\</span>
<span class="nt">--build-arg</span> <span class="nv">VCS_REF</span><span class="o">=</span><span class="si">$(</span>git rev-parse <span class="nt">--short</span> HEAD<span class="si">)</span> <span class="se">\</span>
<span class="nt">--build-arg</span> <span class="nv">BUILD_DATE</span><span class="o">=</span><span class="si">$(</span><span class="nb">date</span> <span class="nt">-u</span> +<span class="s2">"%Y-%m-%dT%H:%M:%SZ"</span><span class="si">)</span> <span class="se">\</span>
<span class="nt">-t</span> stoutyhk/nextcloud-client <span class="nb">.</span> | <span class="nb">tee </span>build.log
docker images stoutyhk/nextcloud-client
</code></pre></div></div>
<p>You can see my current image is the one from DockerHub:</p>
<p><a href="/assets/images/posts/nextcloud-sync/docker-ps.png"><img data-src="/assets/images/posts/nextcloud-sync/docker-ps.png" class="lazyload align-center blur-up" alt="docker-ps screenshot" /></a></p>
<p>The build output:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>root@dusky2: /volume1/homes/james/src/nextcloud-client-docker on master[!?]
<span class="nv">$ </span>./build.sh
Sending build context to Docker daemon 28.67kB
Step 1/19 : FROM alpine:latest
<span class="nt">---</span><span class="o">></span> a187dde48cd2
Step 2/19 : LABEL <span class="nv">maintainer</span><span class="o">=</span><span class="s2">"stoutyhk"</span> <span class="nv">version</span><span class="o">=</span><span class="s2">"0.1"</span> <span class="nv">description</span><span class="o">=</span><span class="s2">"nextcloud sync client"</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>d7bcd965c0d4
Removing intermediate container d7bcd965c0d4
<span class="nt">---</span><span class="o">></span> b6a1c94582c9
Step 3/19 : LABEL based-on<span class="o">=</span><span class="s2">"https://github.com/juanitomint/nextcloud-client-docker"</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>fba05915652c
Removing intermediate container fba05915652c
<span class="nt">---</span><span class="o">></span> 92e4f3b6aba1
Step 4/19 : LABEL <span class="nv">repo</span><span class="o">=</span><span class="s2">"https://github.com/jamesstout/nextcloud-client-docker"</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>63258ff88545
Removing intermediate container 63258ff88545
<span class="nt">---</span><span class="o">></span> 07233dc234c6
Step 5/19 : ARG VCS_REF
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>f03e352100f4
Removing intermediate container f03e352100f4
<span class="nt">---</span><span class="o">></span> 172784412e92
Step 6/19 : ARG BUILD_DATE
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>80600d7575e4
Removing intermediate container 80600d7575e4
<span class="nt">---</span><span class="o">></span> 77bc2f5d481a
Step 7/19 : LABEL vcs-ref<span class="o">=</span><span class="nv">$VCS_REF</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>c58cdd8b58f6
Removing intermediate container c58cdd8b58f6
<span class="nt">---</span><span class="o">></span> 2e598c72729a
Step 8/19 : LABEL build-date<span class="o">=</span><span class="nv">$BUILD_DATE</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>d51c564b1354
Removing intermediate container d51c564b1354
<span class="nt">---</span><span class="o">></span> 2cf77c7ccd0f
Step 9/19 : ARG <span class="nv">USER</span><span class="o">=</span>ncsync
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>10c452b72d23
Removing intermediate container 10c452b72d23
<span class="nt">---</span><span class="o">></span> 62d6402dfe64
Step 10/19 : ARG <span class="nv">GROUP</span><span class="o">=</span><span class="nb">users</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>2c7c84cca017
Removing intermediate container 2c7c84cca017
<span class="nt">---</span><span class="o">></span> 32af7f7e31a6
Step 11/19 : ARG <span class="nv">USER_UID</span><span class="o">=</span>1000
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>a76b14779d6e
Removing intermediate container a76b14779d6e
<span class="nt">---</span><span class="o">></span> d16b1dc7fda7
Step 12/19 : ARG <span class="nv">USER_GID</span><span class="o">=</span>100
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>16a3579c0537
Removing intermediate container 16a3579c0537
<span class="nt">---</span><span class="o">></span> b06b174ac1cb
Step 13/19 : ENV <span class="nv">USER</span><span class="o">=</span><span class="nv">$USER</span> <span class="nv">GROUP</span><span class="o">=</span><span class="nv">$GROUP</span> <span class="nv">USER_UID</span><span class="o">=</span><span class="nv">$USER_UID</span> <span class="nv">USER_GID</span><span class="o">=</span><span class="nv">$USER_GID</span> <span class="nv">NC_USER</span><span class="o">=</span>username <span class="nv">NC_PASS</span><span class="o">=</span>password <span class="nv">NC_INTERVAL</span><span class="o">=</span>500 <span class="nv">NC_URL</span><span class="o">=</span><span class="s2">""</span> <span class="nv">NC_TRUST_CERT</span><span class="o">=</span><span class="nb">false </span><span class="nv">NC_SILENT</span><span class="o">=</span><span class="nb">false </span><span class="nv">NC_EXIT</span><span class="o">=</span><span class="nb">false </span><span class="nv">NC_HIDDEN</span><span class="o">=</span><span class="nb">false </span><span class="nv">NC_MAX_SYNC_RETRIES</span><span class="o">=</span>3
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>85c579929ce1
Removing intermediate container 85c579929ce1
<span class="nt">---</span><span class="o">></span> 82c168659c42
Step 14/19 : RUN adduser <span class="nt">-G</span> <span class="nv">$GROUP</span> <span class="nt">-D</span> <span class="nt">-u</span> <span class="nv">$USER_UID</span> <span class="nv">$USER</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>84755af084c9
Removing intermediate container 84755af084c9
<span class="nt">---</span><span class="o">></span> 9888ccca2dcc
Step 15/19 : RUN apk update <span class="o">&&</span> apk add nextcloud-client <span class="o">&&</span> <span class="nb">rm</span> <span class="nt">-rf</span> /etc/apk/cache
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>702eef96b765
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
v3.11.6-32-g9ddc349524 <span class="o">[</span>http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
v3.11.6-28-g4b76c8208f <span class="o">[</span>http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
OK: 11273 distinct packages available
<span class="o">(</span>1/95<span class="o">)</span> Installing dbus-libs <span class="o">(</span>1.12.16-r2<span class="o">)</span>
<span class="o">(</span>2/95<span class="o">)</span> Installing libgcc <span class="o">(</span>9.2.0-r4<span class="o">)</span>
<span class="o">(</span>3/95<span class="o">)</span> Installing libffi <span class="o">(</span>3.2.1-r6<span class="o">)</span>
<span class="nt">-----8</span><<span class="nt">---snip---8</span><<span class="nt">----</span>
<span class="o">(</span>95/95<span class="o">)</span> Installing nextcloud-client <span class="o">(</span>2.6.1-r0<span class="o">)</span>
^@Executing busybox-1.31.1-r9.trigger
OK: 300 MiB <span class="k">in </span>109 packages
Removing intermediate container 702eef96b765
<span class="nt">---</span><span class="o">></span> 3147165b6612
Step 16/19 : ADD run.sh /usr/bin/run.sh
<span class="nt">---</span><span class="o">></span> 11a9c630e797
Step 17/19 : VOLUME <span class="o">[</span> <span class="s2">"/media/nextcloud"</span> <span class="o">]</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>69454b15f6b6
Removing intermediate container 69454b15f6b6
<span class="nt">---</span><span class="o">></span> af98dc485510
Step 18/19 : VOLUME <span class="o">[</span> <span class="s2">"/config"</span> <span class="o">]</span>
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>94ac262e01a9
Removing intermediate container 94ac262e01a9
<span class="nt">---</span><span class="o">></span> b964f6b7c97c
Step 19/19 : CMD /usr/bin/run.sh
<span class="nt">---</span><span class="o">></span> Running <span class="k">in </span>942c9ddf5ddd
Removing intermediate container 942c9ddf5ddd
<span class="nt">---</span><span class="o">></span> aa064981bb5b
Successfully built aa064981bb5b
Successfully tagged stoutyhk/nextcloud-client:latest
real 5m50.592s
user 0m0.096s
sys 0m0.047s
REPOSITORY TAG IMAGE ID CREATED SIZE
stoutyhk/nextcloud-client latest aa064981bb5b 1 second ago 311MB
stoutyhk/nextcloud-client <none> 6ca8291dece4 3 weeks ago 311MB
</code></pre></div></div>
<p>You can see the <code class="language-plaintext highlighter-rouge">latest</code> tag is now pointing to image <code class="language-plaintext highlighter-rouge">aa064981bb5b</code>. To update, you export the settings, stop the container, <code class="language-plaintext highlighter-rouge">docker rm stoutyhk-nextcloud-client</code>, then import the settings and launch the new image:</p>
<p><a href="/assets/images/posts/nextcloud-sync/new-image.png"><img data-src="/assets/images/posts/nextcloud-sync/new-image.png" class="lazyload align-center blur-up" alt="new-image screenshot" /></a></p>
<p>Spot of housekeeping:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>root@dusky2: /usr/local/bin
<span class="nv">$ </span><span class="k">for </span>image <span class="k">in</span> <span class="si">$(</span>docker images <span class="nt">-f</span> <span class="s2">"dangling=true"</span> | <span class="nb">grep </span>stouty | <span class="nb">grep</span> <span class="nt">-vE</span> <span class="s1">'IMAGE'</span> | <span class="nb">awk</span> <span class="s1">'{ print $3 }'</span><span class="si">)</span><span class="p">;</span> <span class="k">do
</span>docker rmi <span class="s2">"</span><span class="nv">$image</span><span class="s2">"</span><span class="p">;</span>
<span class="k">done
</span>Untagged: stoutyhk/nextcloud-client@sha256:024ba23cc5e182aedb3d30df1bc4e31127fe21f94b35aed55804989b8e154a98
Deleted: sha256:6ca8291dece42c3dda0cf2b6601e0ff1180cf854909b5f3474710ca84babaf04
Deleted: sha256:be8054c4d017455ca6b4afd7e00fb7da03e58beae4a3705575aabcff8af23fb9
Deleted: sha256:c9eae4086cf6c94c60590bb909fbffa90f166371d9c13a53a4a988d48747cdcc
Deleted: sha256:688a57e452c154219a34c88e18ec695ef3a15ce4a1d4f73875d56ff40a8a6de6
</code></pre></div></div>
<p>I have a script that runs weekly to automate pulling of new images:</p>
<script src="https://gist.github.com/jamesstout/c19c227d1ac7dd37108ecbbf5c52eea7.js"></script>
<p>I then need to do the export, rm, import manually as I haven’t figured out how to automate that with Synology Docker package yet.</p>
<p><a href="https://github.com/wagoodman/dive">Dive</a> is a nice tool for examining Docker <a href="https://docs.docker.com/storage/storagedriver/#images-and-layers">images and layers</a>.</p>
<p>You can see what each command in the Dockerfile does to the image. It also gives you an image efficiency estimate:</p>
<blockquote>
<p>Image Efficiency</p>
<p>The lower left pane shows basic layer info and an experimental metric that will guess how much wasted space your image contains. This might be from duplicating files across layers, moving files across layers, or not fully removing files. Both a percentage “score” and total wasted file space is provided.</p>
</blockquote>
<p>Mine is fairly good:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Total Image size: 311MB
Potential wasted space: 179kB
Image efficiency score: 99 %
</code></pre></div></div>
<p>Here’s a look at <code class="language-plaintext highlighter-rouge">dive</code>:</p>
<p><a href="https://asciinema.org/a/2xQFxw4LdqjYuh9cc7WEv4Xc7"><img src="https://asciinema.org/a/2xQFxw4LdqjYuh9cc7WEv4Xc7.svg" alt="asciicast" /></a></p>James StoutSyncing to Nextcloud from a Docker container, and how to build that container.HKWarnings v5.0.02020-05-04T23:40:49+08:002020-05-04T23:40:49+08:00https://stouty.xyz/ios/2020/05/04/hkwarnings<p>When I initially made HKWarnings <a href="https://help.apple.com/xcode/mac/current/#/deve69552ee5">universal</a> and added an iPad interface, I based the interface on a <a href="https://developer.apple.com/library/archive/samplecode/MultipleDetailViews/Introduction/Intro.html">MultipleDetailViews</a> demo from Apple. That version is updated, I used the older code from <a href="https://developer.apple.com/library/archive/samplecode/MultipleDetailViews/History/History.html#//apple_ref/doc/uid/DTS40009775-RevisionHistory-DontLinkElementID_1">2010</a>, so it’s 10 years old and built for iOS 3.2.</p>
<p>Architecturally, there’s a Master ViewController that displays multiple different Detail ViewControllers.</p>
<p>The Master ViewController is a TableView that’s usually hidden:</p>
<p><img data-src="/assets/images/posts/hkwarnings/master.png" class="lazyload blur-up align-center" alt="master vc" /></p>
<p>It’s used to choose one of many Detail VCs:</p>
<figure class="third ">
<a href="/assets/images/posts/hkwarnings/Four-Warning.png" title="Four Warnings">
<img src="/assets/images/posts/hkwarnings/Four-Warning-th.png" alt="Four Warnings" />
</a>
<a href="/assets/images/posts/hkwarnings/Deets-Warning.png" title="Warning Details">
<img src="/assets/images/posts/hkwarnings/Deets-Warning-th.png" alt="Warning Details" />
</a>
<a href="/assets/images/posts/hkwarnings/Radar.png" title="Radar">
<img src="/assets/images/posts/hkwarnings/Radar-th.png" alt="Radar" />
</a>
<a href="/assets/images/posts/hkwarnings/Sat.png" title="Sat">
<img src="/assets/images/posts/hkwarnings/Sat-th.png" alt="Sat" />
</a>
<a href="/assets/images/posts/hkwarnings/TC-Warning.png" title="Typhoon">
<img src="/assets/images/posts/hkwarnings/TC-Warning-th.png" alt="Typhoon" />
</a>
<a href="/assets/images/posts/hkwarnings/Warnings-History.png" title="Warnings-History">
<img src="/assets/images/posts/hkwarnings/Warnings-History-th.png" alt="Warnings-History" />
</a>
<a href="/assets/images/posts/hkwarnings/Weather-Images.png" title="Weather Images">
<img src="/assets/images/posts/hkwarnings/Weather-Images-th.png" alt="Weather Images" />
</a>
<a href="/assets/images/posts/hkwarnings/Weather-Image.png" title="Weather Image">
<img src="/assets/images/posts/hkwarnings/Weather-Image-th.png" alt="Weather Image" />
</a>
<a href="/assets/images/posts/hkwarnings/Deets-Warning-Screen-CN.png" title="Warning Details Chinese">
<img src="/assets/images/posts/hkwarnings/Deets-Warning-Screen-CN-th.png" alt="Warning Details Chinese" />
</a>
<figcaption>Click a row in the Master table and the selected Detail VC loads.
</figcaption>
</figure>
<p>Anyway, this 10 year old code has survived <a href="https://en.wikipedia.org/wiki/IOS_version_history">9 OS updates</a>, but finally, on iOS 13, it no longer works. The code below that switches the Detail VC and hides the Master <em>is</em> called, but the <code class="language-plaintext highlighter-rouge">UIPopoverController</code> is always <code class="language-plaintext highlighter-rouge">nil</code>:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">-</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="nf">splitViewController</span><span class="p">:(</span><span class="n">UISplitViewController</span> <span class="o">*</span><span class="p">)</span><span class="nv">svc</span>
<span class="nf">willHideViewController</span><span class="p">:(</span><span class="n">UIViewController</span> <span class="o">*</span><span class="p">)</span><span class="nv">aViewController</span>
<span class="nf">withBarButtonItem</span><span class="p">:(</span><span class="n">UIBarButtonItem</span> <span class="o">*</span><span class="p">)</span><span class="nv">barButtonItem</span>
<span class="nf">forPopoverController</span><span class="p">:(</span><span class="n">UIPopoverController</span> <span class="o">*</span><span class="p">)</span><span class="nv">pc</span> <span class="p">{</span>
<span class="c1">// Keep references to the popover controller and the popover button,</span>
<span class="c1">// and tell the detail view controller to show the button.</span>
<span class="n">barButtonItem</span><span class="p">.</span><span class="n">title</span> <span class="o">=</span> <span class="n">NSLocalizedString</span><span class="p">(</span><span class="s">@"Menu"</span><span class="p">,</span> <span class="s">"Menu"</span><span class="p">);</span>
<span class="n">self</span><span class="p">.</span><span class="n">popoverController</span> <span class="o">=</span> <span class="n">pc</span><span class="p">;</span>
<span class="n">self</span><span class="p">.</span><span class="n">rootPopoverButtonItem</span> <span class="o">=</span> <span class="n">barButtonItem</span><span class="p">;</span>
<span class="n">UIViewController</span> <span class="o"><</span><span class="n">SubstitutableDetailViewController</span><span class="o">></span> <span class="o">*</span><span class="n">detailViewController</span> <span class="o">=</span> <span class="p">[</span><span class="n">splitViewController</span><span class="p">.</span><span class="n">viewControllers</span> <span class="nf">objectAtIndex</span><span class="p">:</span><span class="mi">1</span><span class="p">];</span>
<span class="p">[</span><span class="n">detailViewController</span> <span class="nf">showRootPopoverButtonItem</span><span class="p">:</span><span class="n">rootPopoverButtonItem</span><span class="p">];</span>
<span class="p">}</span>
<span class="k">-</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="nf">splitViewController</span><span class="p">:(</span><span class="n">UISplitViewController</span> <span class="o">*</span><span class="p">)</span><span class="nv">svc</span>
<span class="nf">willShowViewController</span><span class="p">:(</span><span class="n">UIViewController</span> <span class="o">*</span><span class="p">)</span><span class="nv">aViewController</span>
<span class="nf">invalidatingBarButtonItem</span><span class="p">:(</span><span class="n">UIBarButtonItem</span> <span class="o">*</span><span class="p">)</span><span class="nv">barButtonItem</span> <span class="p">{</span>
<span class="c1">// Nil out references to the popover controller and the popover button,</span>
<span class="c1">// and tell the detail view controller to hide the button.</span>
<span class="n">UIViewController</span> <span class="o"><</span><span class="n">SubstitutableDetailViewController</span><span class="o">></span> <span class="o">*</span><span class="n">detailViewController</span> <span class="o">=</span> <span class="p">[</span><span class="n">splitViewController</span><span class="p">.</span><span class="n">viewControllers</span> <span class="nf">objectAtIndex</span><span class="p">:</span><span class="mi">1</span><span class="p">];</span>
<span class="p">[</span><span class="n">detailViewController</span> <span class="nf">invalidateRootPopoverButtonItem</span><span class="p">:</span><span class="n">rootPopoverButtonItem</span><span class="p">];</span>
<span class="n">self</span><span class="p">.</span><span class="n">popoverController</span> <span class="o">=</span> <span class="nb">nil</span><span class="p">;</span>
<span class="n">self</span><span class="p">.</span><span class="n">rootPopoverButtonItem</span> <span class="o">=</span> <span class="nb">nil</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Nothing I tried could make it work. There are not too many other people searching for an answer on Stackoverflow. I started a major rewrite based on the stock Xcode Master-Detail project:</p>
<figure class="half ">
<a href="/assets/images/posts/hkwarnings/master-detail.png" title="Four Warning">
<img src="/assets/images/posts/hkwarnings/master-detail-th.png" alt="Four Warnings" />
</a>
<a href="/assets/images/posts/hkwarnings/master-detail-code.png" title="Warning Details">
<img src="/assets/images/posts/hkwarnings/master-detail-code-th.png" alt="Warning Details" />
</a>
</figure>
<p>But it’s all <a href="https://developer.apple.com/library/archive/documentation/ToolsLanguages/Conceptual/Xcode_Overview/DesigningwithStoryboards.html#//apple_ref/doc/uid/TP40010215-CH43-SW1">Storyboards, scenes and segues</a>. I’m not a fan. I soon had a huge mess of a main storyboard. I prefer to do things <a href="https://www.toptal.com/ios/ios-user-interfaces-storyboards-vs-nibs-vs-custom-code">programmatically</a>, well, a mix of code and <a href="http://www.monobjc.net/xib-file-format.html">xibs</a>.</p>
<p>Eventually I found <a href="https://github.com/mutualmobile/MMDrawerController">MMDrawerController</a>, it’s also fairly old code (targets iOS 7), but I quickly built a working prototype that worked on iOS 13.</p>
<p>It took a few days, but I have just released <a href="https://apps.apple.com/us/app/hkwarnings/id370901118#?platform=ipad">v5.0.0 of HKWarnings</a>, using the new side drawer.</p>
<h2 id="code-changes">Code Changes</h2>
<p>178 files were changed:</p>
<p><a href="/assets/images/posts/hkwarnings/changes.png"><img data-src="/assets/images/posts/hkwarnings/changes.png" src="/assets/images/posts/hkwarnings/changes-lq.png" class="lazyload blur-up" alt="Git files changes screenshot" /></a></p>
<p>Lines changed:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~/Projects/iPhone Apps/HKWarnings on 5.0.1[!?<span class="nv">$]</span>
<span class="nv">$ </span>git log <span class="nt">--numstat</span> <span class="nt">--pretty</span><span class="o">=</span><span class="s2">"%H"</span> <span class="nt">--author</span><span class="o">=</span><span class="s2">"James Stout"</span> <span class="se">\</span>
<span class="o">></span> b318bae1 bc535bb7 | <span class="nb">grep</span> <span class="nt">-v</span> Pods | <span class="nb">grep</span> <span class="nt">-v</span> Carthage <span class="se">\</span>
<span class="o">></span> | <span class="nb">awk</span> <span class="s1">'NF==3 {plus+=$1; minus+=$2} END {printf("+%d, -%d\n", plus, minus)}'</span>
+183978, <span class="nt">-119453</span>
</code></pre></div></div>
<p><strong>+183978, -119453</strong></p>
<p>BTW, that command takes a <em>quite</em> a while:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~/Projects/iPhone Apps/HKWarnings on 5.0.1[!?<span class="nv">$]</span>
<span class="nv">$ </span><span class="nb">time </span>git log <span class="nt">--numstat</span> <span class="nt">--pretty</span><span class="o">=</span><span class="s2">"%H"</span> <span class="nt">--author</span><span class="o">=</span><span class="s2">"James Stout"</span> <span class="se">\</span>
<span class="o">></span> b318bae1 bc535bb7 | <span class="nb">grep</span> <span class="nt">-v</span> Pods | <span class="nb">grep</span> <span class="nt">-v</span> Carthage <span class="se">\</span>
<span class="o">></span> | <span class="nb">awk</span> <span class="s1">'NF==3 {plus+=$1; minus+=$2} END {printf("+%d, -%d\n", plus, minus)}'</span>
+183978, <span class="nt">-119453</span>
real 0m40.285s
user 0m35.363s
sys 0m4.840s
</code></pre></div></div>
<h2 id="some-stats">Some Stats</h2>
<p><a href="https://github.com/AlDanial/cloc/">cloc</a> output, excluding Pods:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~/Projects/iPhone Apps/HKWarnings on 5.0.1[!?<span class="nv">$]</span>
<span class="nv">$ </span>cloc <span class="nt">--ignored</span><span class="o">=</span>ignored.txt <span class="se">\</span>
<span class="o">></span> <span class="nt">--exclude-dir</span><span class="o">=</span>Pods,HKWeatherWarningsTabs.xcodeproj,HKWeatherWarningsTabs.xcworkspace <span class="se">\</span>
<span class="o">></span> <span class="nt">--exclude-ext</span><span class="o">=</span>entitlements,plist,strings,xcodeproj <span class="se">\</span>
<span class="o">></span> <span class="nt">--report-file</span><span class="o">=</span>cloc-report3.txt <span class="nb">.</span>
394 text files.
380 unique files.
Wrote ignored.txt
44 files ignored.
Wrote cloc-report3.txt
github.com/AlDanial/cloc v 1.84 <span class="nv">T</span><span class="o">=</span>0.40 s <span class="o">(</span>873.2 files/s, 133349.8 lines/s<span class="o">)</span>
<span class="nt">--------------------------------------------------------------------------------</span>
Language files blank comment code
<span class="nt">--------------------------------------------------------------------------------</span>
Objective C 91 11963 5279 20095
XML 41 105 5 3536
Bourne Shell 12 83 80 3522
JSON 99 1 0 3227
C/C++ Header 96 1073 1060 2157
Swift 2 108 40 448
HTML 1 19 0 420
Bourne Again Shell 7 44 11 146
Perl 1 21 121 30
YAML 1 0 0 7
<span class="nt">--------------------------------------------------------------------------------</span>
SUM: 351 13417 6596 33588
<span class="nt">--------------------------------------------------------------------------------</span>
</code></pre></div></div>
<p>Including Pods:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~/Projects/iPhone Apps/HKWarnings on 5.0.1[!?<span class="nv">$]</span>
<span class="nv">$ </span>cloc <span class="nt">--ignored</span><span class="o">=</span>ignored.txt <span class="se">\</span>
<span class="o">></span> <span class="nt">--exclude-dir</span><span class="o">=</span>HKWeatherWarningsTabs.xcodeproj,HKWeatherWarningsTabs.xcworkspace <span class="se">\</span>
<span class="o">></span> <span class="nt">--exclude-ext</span><span class="o">=</span>entitlements,plist,strings,xcodeproj <span class="se">\</span>
<span class="nt">--report-file</span><span class="o">=</span>cloc-inc-pods-report3.txt <span class="nb">.</span>
1645 text files.
1101 unique files.
Wrote ignored.txt
697 files ignored.
Wrote cloc-inc-pods-report3.txt
github.com/AlDanial/cloc v 1.84 <span class="nv">T</span><span class="o">=</span>1.25 s <span class="o">(</span>758.6 files/s, 101853.4 lines/s<span class="o">)</span>
<span class="nt">--------------------------------------------------------------------------------</span>
Language files blank comment code
<span class="nt">--------------------------------------------------------------------------------</span>
Objective C 303 18922 11355 48001
C/C++ Header 443 5716 14128 9569
Bourne Shell 15 124 149 3804
XML 41 105 5 3536
Markdown 32 1410 0 3449
JSON 100 1 0 3255
C 3 379 261 1834
Swift 2 108 40 448
HTML 1 19 0 420
Bourne Again Shell 7 44 11 146
Perl 1 21 121 30
YAML 1 0 0 7
<span class="nt">--------------------------------------------------------------------------------</span>
SUM: 949 26849 26070 74499
<span class="nt">--------------------------------------------------------------------------------</span>
</code></pre></div></div>
<p>For reference, these are the <a href="https://cocoapods.org/">Pods</a> I use:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">source</span> <span class="s1">'https://github.com/CocoaPods/Specs.git'</span>
<span class="n">source</span> <span class="s1">'https://github.com/jamesstout/Specs.git'</span>
<span class="n">platform</span> <span class="ss">:ios</span><span class="p">,</span> <span class="s1">'11.0'</span>
<span class="n">target</span> <span class="ss">:HKWeatherWarningsTabs</span> <span class="k">do</span>
<span class="n">pod</span> <span class="s1">'OneSignal'</span>
<span class="n">pod</span> <span class="s1">'Fabric'</span>
<span class="n">pod</span> <span class="s1">'Crashlytics'</span><span class="p">,</span> <span class="s1">'~> 3.12'</span>
<span class="n">pod</span> <span class="s1">'MBProgressHUD'</span><span class="p">,</span> <span class="ss">:git</span> <span class="o">=></span> <span class="s1">'https://github.com/jamesstout/MBProgressHUD.git'</span><span class="p">,</span> <span class="ss">:branch</span> <span class="o">=></span> <span class="s1">'pre-iOS13'</span>
<span class="n">pod</span> <span class="s1">'StandardPaths'</span>
<span class="n">pod</span> <span class="s1">'SDWebImage'</span><span class="p">,</span> <span class="s1">'~> 4.3'</span>
<span class="n">pod</span> <span class="s1">'SDWebImage/GIF'</span>
<span class="n">pod</span> <span class="s1">'SAMCategories'</span>
<span class="n">pod</span> <span class="s1">'TSMarkdownParser'</span><span class="p">,</span> <span class="s1">'~> 2.1'</span>
<span class="n">pod</span> <span class="s1">'PureLayout'</span>
<span class="n">pod</span> <span class="s1">'FlatUIKit'</span>
<span class="n">pod</span> <span class="s1">'KVOController'</span>
<span class="n">pod</span> <span class="s1">'DTTJailbreakDetection'</span>
<span class="n">pod</span> <span class="s1">'OHAttributedStringAdditions'</span>
<span class="n">pod</span> <span class="s1">'JSQSystemSoundPlayer'</span>
<span class="n">pod</span> <span class="s1">'Google-Mobile-Ads-SDK'</span>
<span class="n">pod</span> <span class="s1">'NYTPhotoViewer'</span><span class="p">,</span> <span class="s1">'~> 1.2.0'</span>
<span class="n">pod</span> <span class="s1">'DateTools'</span>
<span class="n">pod</span> <span class="s1">'HVTableView'</span>
<span class="n">pod</span> <span class="s1">'CocoaLumberjack'</span><span class="p">,</span> <span class="s1">'~> 3.6.1'</span>
<span class="n">pod</span> <span class="s1">'CompressingAndUploadingLogFileManager'</span>
<span class="n">pod</span> <span class="s1">'GZIP'</span><span class="p">,</span> <span class="s1">'~> 1.3.0'</span>
<span class="n">pod</span> <span class="s1">'MMDrawerController'</span>
<span class="n">target</span> <span class="ss">:HKWarningsTests</span> <span class="k">do</span>
<span class="n">inherit!</span> <span class="ss">:search_paths</span>
<span class="n">pod</span> <span class="s1">'Expecta'</span><span class="p">,</span> <span class="s1">'~> 1.0'</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="significant-pods">Significant Pods</h2>
<ul>
<li><a href="https://www.onesignal.com/">OneSignal</a> - for push notifications.</li>
<li><a href="https://fabric.io/">Fabric/Crashlytics</a> - for crash reporting, moving to <a href="https://firebase.google.com/">Firebase</a> soon.</li>
<li><a href="https://github.com/matej/MBProgressHUD">MBProgressHUD</a> - an iOS activity indicator view, but pointing to a branch on my fork, so it <a href="https://github.com/jamesstout/MBProgressHUD/commit/c013d38c2dee75f52245290c394c23a68c15d506">compiles on iOS 12.1</a>.</li>
<li><a href="https://github.com/SDWebImage/SDWebImage">SDWebImage</a> - an asynchronous image downloader with cache support.</li>
<li><a href="https://github.com/PureLayout/PureLayout">PureLayout</a> - API for iOS & OS X Auto Layout. None of that <a href="https://developer.apple.com/xcode/interface-builder/">IB</a> rubbish.</li>
<li><a href="https://github.com/CocoaLumberjack/CocoaLumberjack">CocoaLumberjack</a> - a fast & simple, yet powerful & flexible logging framework.</li>
<li><a href="https://github.com/jamesstout/CompressingAndUploadingLogFileManager">CompressingAndUploadingLogFileManager</a> - my, as yet unpublished<sup id="fnref:fn-repo" role="doc-noteref"><a href="#fn:fn-repo" class="footnote" rel="footnote">1</a></sup> library, that compresses log files and then uploads them to an HTTP server for examination, stats etc.</li>
<li><a href="https://github.com/nicklockwood/GZIP">GZIP</a> - to unzip <code class="language-plaintext highlighter-rouge">.gz</code> files. To <a href="https://developer.apple.com/documentation/xcode/reducing_your_app_s_size/doing_basic_optimization_to_reduce_your_app_s_size?language=objc">reduce package size</a> I zip some text files. See below.</li>
<li><a href="https://github.com/laptobbe/TSMarkdownParser">TSMarkdownParser</a> - a markdown to NSAttributedString parser. I use it to display the <a href="https://github.com/CocoaPods/CocoaPods/wiki/Acknowledgements">acknowledgements</a> Markdown created by the <code class="language-plaintext highlighter-rouge">pod install</code> command. Again, see below.</li>
</ul>
<p>In my Podfile, I have a <code class="language-plaintext highlighter-rouge">post_install</code> hook that does various things, including, copying the acknowledgements <code class="language-plaintext highlighter-rouge">plist</code> to the Settings bundle, and the Markdown to a text file that I then compress.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'fileutils'</span>
<span class="no">FileUtils</span><span class="p">.</span><span class="nf">cp_r</span><span class="p">(</span><span class="s1">'Pods/Target Support Files/Pods-HKWeatherWarningsTabs/Pods-HKWeatherWarningsTabs-acknowledgements.plist'</span><span class="p">,</span> <span class="s1">'Settings.bundle/Acknowledgements.plist'</span><span class="p">,</span> <span class="ss">:remove_destination</span> <span class="o">=></span> <span class="kp">true</span><span class="p">)</span>
<span class="no">FileUtils</span><span class="p">.</span><span class="nf">cp_r</span><span class="p">(</span><span class="s1">'Pods/Target Support Files/Pods-HKWeatherWarningsTabs/Pods-HKWeatherWarningsTabs-acknowledgements.markdown'</span><span class="p">,</span> <span class="s1">'Acknowledgements.txt'</span><span class="p">,</span> <span class="ss">:remove_destination</span> <span class="o">=></span> <span class="kp">true</span><span class="p">)</span>
<span class="nb">require</span> <span class="s1">'zlib'</span>
<span class="n">zipped</span> <span class="o">=</span> <span class="s2">"Acknowledgements.txt.gz"</span>
<span class="no">Zlib</span><span class="o">::</span><span class="no">GzipWriter</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="n">zipped</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">gz</span><span class="o">|</span>
<span class="n">gz</span><span class="p">.</span><span class="nf">write</span> <span class="no">IO</span><span class="p">.</span><span class="nf">binread</span><span class="p">(</span><span class="s2">"Acknowledgements.txt"</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>In the app code to display the Markdown, I first unzip, then parse:</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">NSFileManager</span> <span class="o">*</span><span class="n">fm</span> <span class="o">=</span> <span class="p">[</span><span class="n">NSFileManager</span> <span class="nf">defaultManager</span><span class="p">];</span>
<span class="n">NSString</span> <span class="o">*</span><span class="n">pathGZ</span> <span class="o">=</span> <span class="p">[</span><span class="n">fm</span> <span class="nf">pathForResource</span><span class="p">:</span><span class="s">@"Acknowledgements.txt.gz"</span><span class="p">];</span>
<span class="k">if</span> <span class="p">([</span><span class="n">fm</span> <span class="nf">fileExistsAtPath</span><span class="p">:</span><span class="n">pathGZ</span><span class="p">])</span> <span class="p">{</span>
<span class="n">NSData</span> <span class="o">*</span><span class="n">zippedContent</span> <span class="o">=</span> <span class="p">[</span><span class="n">NSData</span> <span class="nf">dataWithContentsOfFile</span><span class="p">:</span><span class="n">pathGZ</span><span class="p">];</span>
<span class="n">NSData</span> <span class="o">*</span><span class="n">unZippedContent</span> <span class="o">=</span> <span class="nb">nil</span><span class="p">;</span>
<span class="n">NSString</span> <span class="o">*</span><span class="n">content</span> <span class="o">=</span> <span class="nb">nil</span><span class="p">;</span>
<span class="k">if</span><span class="p">([</span><span class="n">zippedContent</span> <span class="nf">isGzippedData</span><span class="p">]){</span>
<span class="n">DDLogDebug</span><span class="p">(</span><span class="s">@"Data is zipped"</span><span class="p">);</span>
<span class="n">unZippedContent</span> <span class="o">=</span> <span class="p">[</span><span class="n">zippedContent</span> <span class="nf">gunzippedData</span><span class="p">];</span>
<span class="n">content</span> <span class="o">=</span> <span class="p">[[</span><span class="n">NSString</span> <span class="nf">alloc</span><span class="p">]</span> <span class="nf">initWithData</span><span class="p">:</span><span class="n">unZippedContent</span> <span class="nf">encoding</span><span class="p">:</span><span class="n">NSUTF8StringEncoding</span><span class="p">];</span>
<span class="p">}</span>
<span class="k">else</span><span class="p">{</span>
<span class="n">DDLogError</span><span class="p">(</span><span class="s">@"Acknowledgements.txt.gz is not zipped"</span><span class="p">);</span>
<span class="n">content</span> <span class="o">=</span> <span class="s">@"ERROR"</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">DDLogDebug</span><span class="p">(</span><span class="s">@"content is %@"</span><span class="p">,</span> <span class="n">content</span><span class="p">);</span>
<span class="n">self</span><span class="p">.</span><span class="n">parser</span> <span class="o">=</span> <span class="p">[</span><span class="n">TSMarkdownParser</span> <span class="nf">standardParser</span><span class="p">];</span>
<span class="n">NSAttributedString</span> <span class="o">*</span><span class="n">result</span> <span class="o">=</span> <span class="p">[</span><span class="n">self</span><span class="p">.</span><span class="n">parser</span> <span class="nf">attributedStringFromMarkdown</span><span class="p">:</span><span class="n">content</span><span class="p">];</span>
<span class="n">_textView</span><span class="p">.</span><span class="n">attributedText</span> <span class="o">=</span> <span class="n">result</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Saves about 47kB<sup id="fnref:fn-gz" role="doc-noteref"><a href="#fn:fn-gz" class="footnote" rel="footnote">2</a></sup>:</p>
<p><a href="/assets/images/posts/hkwarnings/gz.png"><img data-src="/assets/images/posts/hkwarnings/gz.png" class="lazyload align-center blur-up" alt="gzipped ack file" /></a></p>
<p>I do this so that I can display the acknowledgements in the app, and in the OS Settings app:</p>
<figure class="third ">
<a href="/assets/images/posts/hkwarnings/ack-inapp.png" title="acknowledgements in-app">
<img src="/assets/images/posts/hkwarnings/ack-inapp-th.png" alt="ack-inapp.png" />
</a>
<a href="/assets/images/posts/hkwarnings/ack-settings.png" title="acknowledgements in setting">
<img src="/assets/images/posts/hkwarnings/ack-settings-th.png" alt="acknowledgements in settings" />
</a>
<a href="/assets/images/posts/hkwarnings/ack-settings-1.png" title="acknowledgements in setting">
<img src="/assets/images/posts/hkwarnings/ack-settings-1-th.png" alt="acknowledgements in settings" />
</a>
</figure>
<p>Anyway, here’s v5.0.0 (click through to view the video):</p>
<p><a href="https://www.dropbox.com/s/y4q7u806b5zjezh/HKWarnings.mp4?dl=0"><img data-src="/assets/images/posts/hkwarnings/HKWarnings.png" class="lazyload align-center blur-up" alt="HKWarnings.png" /></a></p>
<hr />
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:fn-repo" role="doc-endnote">
<p>See that private Spec source in the <a href="https://guides.cocoapods.org/using/the-podfile.html">Podfile</a>? <code class="language-plaintext highlighter-rouge">source 'https://github.com/jamesstout/Specs.git'</code> <a href="#fnref:fn-repo" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:fn-gz" role="doc-endnote">
<p>Well, about 30kB as the GZIP library is about 17kB. <a href="#fnref:fn-gz" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>James StoutFixing 10 year old code to build HKWarnings v5.0.0.Update to MobileDevice.framework on macOS Breaks Old GarageBand2020-04-29T19:31:05+08:002020-04-29T19:31:05+08:00https://stouty.xyz/macos/2020/04/29/mobile-device<p>My old iMac isn’t supported by Mojave or Catalina, so I’m still on High Sierra. Whenever I update to a new iOS version, I get a <a href="https://ios.gadgethacks.com/how-to/fix-software-update-is-required-connect-your-iphone-warning-your-mac-0185898/">prompt from iTunes</a> saying <em>A Software Update Is Required to Connect to Your iPhone</em>, and I just blindly install.</p>
<p>However, the latest install seems to have broken GarageBand.</p>
<p><a href="/assets/images/posts/mobile-device/problem.png"><img data-src="/assets/images/posts/mobile-device/problem.png" class="lazyload blur-up" alt="GarageBand Problem" /></a></p>
<p>Running from the command line:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /Applications/GarageBand.app/Contents/MacOS
<span class="nv">$ </span>./GarageBand
dyld: Library not loaded: /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice
Referenced from: /System/Library/Frameworks/CoreAudioKit.framework/Versions/A/CoreAudioKit
Reason: no suitable image found. Did find:
/System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice: mach-o, but wrong architecture
/System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice: <span class="nb">stat</span><span class="o">()</span> failed with <span class="nv">errno</span><span class="o">=</span>45
/System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice: mach-o, but wrong architecture
Abort <span class="nb">trap</span>: 6
</code></pre></div></div>
<p>Hmmm, let’s check out MobileDevice:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A
<span class="nv">$ </span>file MobileDevice
MobileDevice: Mach-O 64-bit dynamically linked shared library x86_64
</code></pre></div></div>
<p>64-bit only. And updated recently:</p>
<p><a href="/assets/images/posts/mobile-device/md.png"><img data-src="/assets/images/posts/mobile-device/md.png" class="lazyload blur-up" alt="MobileDevice file listing" /></a></p>
<p>For some confusion:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~
<span class="nv">$ </span><span class="nb">arch
</span>i386
james@Jamess-iMac: ~
<span class="nv">$ </span><span class="nb">uname</span> <span class="nt">-m</span>
x86_64
james@Jamess-iMac: ~
<span class="nv">$ </span><span class="nb">uname</span> <span class="nt">-p</span>
i386
james@Jamess-iMac: ~
<span class="nv">$ </span>machine
i486
</code></pre></div></div>
<p>Anyway, my old version of GarageBand:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: ~
<span class="nv">$ </span>/usr/bin/file <span class="s1">'/Applications/GarageBand.app/Contents/MacOS/GarageBand'</span>
/Applications/GarageBand.app/Contents/MacOS/GarageBand: Mach-O executable i386
</code></pre></div></div>
<p>Hmmm, only 32-bit Intel code.</p>
<p>I have a clone of my drive from before the update, let’s check out MobileDevice from back then:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /Volumes/CCC/System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A
<span class="nv">$ </span>file MobileDevice
MobileDevice: Mach-O universal binary with 2 architectures: <span class="o">[</span>i386:Mach-O dynamically linked shared library i386] <span class="o">[</span>x86_64]
MobileDevice <span class="o">(</span><span class="k">for </span>architecture i386<span class="o">)</span>: Mach-O dynamically linked shared library i386
MobileDevice <span class="o">(</span><span class="k">for </span>architecture x86_64<span class="o">)</span>: Mach-O 64-bit dynamically linked shared library x86_64
</code></pre></div></div>
<p>Built for both archs…</p>
<p>Sooooo, I could fiddle with <code class="language-plaintext highlighter-rouge">csrutil</code> and <a href="https://forums.creativecow.net/thread/19/897177#897188">replace the new framework with the one from my clone</a>, or, see if there’s an update. No update has ever shown up in the App Store for GarageBand, but it turns out there is one:</p>
<p><a href="/assets/images/posts/mobile-device/gb-update.png"><img data-src="/assets/images/posts/mobile-device/gb-update.png" src="/assets/images/posts/mobile-device/gb-update-lq.png" class="lazyload blur-up" alt="GarageBand update" /></a></p>
<p>After the update:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>james@Jamess-iMac: /Applications/GarageBand.app/Contents/MacOS
<span class="nv">$ </span>file GarageBand
GarageBand: Mach-O 64-bit executable x86_64
</code></pre></div></div>
<p>And it works:</p>
<p><a href="/assets/images/posts/mobile-device/new-gb.png"><img data-src="/assets/images/posts/mobile-device/new-gb.png" src="/assets/images/posts/mobile-device/new-gb-lq.png" class="lazyload blur-up" alt="GarageBand latest version running" /></a></p>
<p>Um, if you want to check out the song I recorded, here’s a preview:</p>
<p>/assets/images/posts/mobile-device/shallow-preview.mp3</p>James StoutiTunes update breaks GarageBand.