Progress towards background work replacement, work getting new NetworkManager and HttpClient up and running

Gemfile.lock

@@ -1,13 +1,13 @@
 GEM
   remote: https://rubygems.org/
   specs:
-    async (2.35.2)
+    async (2.36.0)
       console (~> 1.29)
       fiber-annotation
       io-event (~> 1.11)
       metrics (~> 0.12)
       traces (~> 0.18)
-    async-http (0.94.0)
+    async-http (0.94.2)
       async (>= 2.10.2)
       async-pool (~> 0.11)
       io-endpoint (~> 0.14)
@@ -30,7 +30,7 @@ GEM
       fiber-annotation
       fiber-local (~> 1.1)
       json
-    cyberarm_engine (0.25.0)
+    cyberarm_engine (0.25.1)
       gosu (~> 1.1)
     digest-crc (0.7.0)
       rake (>= 12.0.0, < 14.0.0)
@@ -43,7 +43,7 @@ GEM
     fiber-storage (1.0.1)
     fiddle (1.1.8)
     gosu (1.4.6)
-    io-endpoint (0.16.0)
+    io-endpoint (0.17.2)
     io-event (1.14.2)
     io-stream (0.11.1)
     ircparser (1.0.0)
@@ -52,8 +52,8 @@ GEM
       fiddle
     metrics (0.15.0)
     protocol-hpack (1.5.1)
-    protocol-http (0.58.0)
+    protocol-http (0.58.1)
-    protocol-http1 (0.36.0)
+    protocol-http1 (0.37.0)
       protocol-http (~> 0.58)
     protocol-http2 (0.24.0)
       protocol-hpack (~> 1.4)
@@ -97,4 +97,4 @@ DEPENDENCIES
   win32-security
 
 BUNDLED WITH
-   2.6.8
+  4.0.3

docs/README.md

@@ -0,0 +1,35 @@
+# Documentation
+
+## File Formats
+
+Documentation and examples of file formats.
+
+### [Legacy Manifest](file_formats/LEGACY_MANIFEST.md)
+
+Blue Hell Productions `manifest.xml`. Still in use by W3D Hub.
+
+### [Megafest](file_formats/MEGAFEST.md)
+
+Prototype new thing `megafest.json`
+
+### [MIX](file_formats/MIX.md)
+
+Westwood archive format `<name>.mix`, `<name>.dat` and `<name>.pkg`
+
+### [W3D Hub Patch](file_formats/W3D_HUB_PATCH.md)
+
+Describes how to update a MIX archive from a package patch
+
+## Protocols
+
+### [LAN Discovery](protocols/LAN_DISCOVERY.md)
+
+How launchers and other applications discovery each other.
+
+### [Launcher Remote](protocols/LAUNCHER_REMOTE.md)
+
+Remotely control and get progress reports from the launcher
+
+### [LAN Package Share](protocols/LAN_PACKAGE_SHARE.md)
+
+Enable launchers on the same network to discover each other and share packages over LAN.

docs/file_formats/LEGACY_MANIFEST.md

@@ -0,0 +1,73 @@
+# Legacy Manifest
+
+In use since Blue Hell Productions
+
+> [!IMPORTANT]
+> References to packages exclude `.zip`
+
+Example `Full` manifest:
+
+```xml
+
+<BHP_Game_Manifest game="apb" version="0.9935.1.0" type="Full">
+    <Dependency name="msvc-2015.zip"/>
+    <Dependency name="directx-43.zip"/>
+    <File name="game.exe" checksum="C7AE972A2B9CF6EFCADE7919323F048F5CFAF2D40BD5A67D5A91F8D8A3759CA4" package="binaries"/>
+    <File name="MemoryManager.dll" checksum="B60A2B3F6A07D729EE5C2902CB9A6BEDAF55EEDB13F8514BAC0AD6F5D51DD0C2" package="binaries"/>
+    <File name="scripts.dll" checksum="76CAAEBF21E4D6A34C3ABDD474D59580DD1EAB097C9585A1FFC0E28FF0FCAF56" package="binaries"/>
+    <File name="Scripts2.dll" checksum="A0439B3BFE6CB497F402291548A312A9F43C0D38BDAF26C83E21DBD3F4D55702" package="binaries"/>
+    <File name="shared.dll" checksum="17A8797EC5F4C34E241681F4AF76EEA93856D6A1854EC9838B263AEC89CDF036" package="binaries"/>
+    <File name="ttle.dll" checksum="C005EB50D77675CE712B30A4402AB0D152D3993EB509E35AEDCB834CED54FADF" package="binaries"/>
+    <File name="ttversion.txt" checksum="17A9E627A03E20AC33849330468BF8A80B48449C9D473DF45F1052A160083167" package="misc"/>
+    <File name="Data/always.dat" checksum="D4D09149C7B5368AB3947AEC7B6E7781208F592FCA4390261F04A8E9C1887EB3" package="always"/>
+    <File name="Data/RA_Volcano.mix" checksum="4F3CB80BCF200659B3B4B167CCB64ECB36A3BA917681F4AB8027568EC187B88D" package="RA_Volcano"/>
+    <File name="Data/Movies/ea_ww.bik" checksum="065B8B0894CF9C1C09A940D26069C54A771EBB3092143BA37E97F8CA85B8CCAD" package="movies"/>
+    <File name="Data/Movies/R_Intro.BIK" checksum="0D9A2B2ABFD9680DF1C8D0F876DA4B60F015CA5C56EC6C0593528A3F88E2A58F" package="movies"/>
+</BHP_Game_Manifest>
+```
+
+Example `Patch` manifest:
+
+```xml
+
+<BHP_Game_Manifest game="apb" version="3.6.4.1" type="Patch" baseVersion="3.6.4.0">
+    <Dependency name="msvc-2022.zip"/>
+    <Dependency name="msvc-2015.zip" removedsince="3.6.4.1"/>
+    <File name="data/Always.dat" checksum="7B27D547E5A50401C9CA2851ACD4AC3F3CDA104F34650E1C32EB63A4120E3C90">
+        <Patch from="3.6.4.0" package="Always.patch.3.6.4.0"/>
+    </File>
+    <File name="data/Always_Emitters.dat" checksum="5C3EA8E5C4D7278F5EF1AF3CE1FC4F698C5A7D1FE7116DA0376A709E7CD3B2AE">
+        <Patch from="3.6.3.0" package="Always_Emitters.patch.3.6.3.0"/>
+    </File>
+    <File name="data/Always_Vehicles.dat" checksum="C6D4F6B3412C26065DE9B9F739BEC041B49FD613DB2686D8F231C2F692B263C5">
+        <Patch from="3.6.4.0" package="Always_Vehicles.patch.3.6.4.0"/>
+    </File>
+    <File name="data/RA_TestUnits.mix" removedsince="3.6.4.1"/>
+    <File name="data/RA_RidgeWar.mix" checksum="C3438C532C27F535CD958BEA9702160E952BC6CE58F9B43778F91DA23C6AE9FE" package="RA_RidgeWar"/>
+    <File name="data/RA_SwampOfIllusions.mix" checksum="A0787979664E5977A1949587382A3A84B9F1526F81934EBB7A3AC9334F9D4059">
+        <Patch from="3.6.4.0" package="RA_SwampOfIllusions.patch.3.6.4.0"/>
+    </File>
+</BHP_Game_Manifest>
+```
+
+## SPECIFICATION
+### BHP_GAME_MANIFEST (Root Element)
+* **GAME** - Unique Application ID
+* **VERSION** - Application Unique Version for Build
+* **TYPE** - Type of Build
+  * **Full** - Full Build
+  * **Patch** - Patch Build
+* **BASEVERSION** - *OPTIONAL* Version this build was based upon 
+
+### DEPENDENCY
+* **NAME** - Package name of dependency sans file extension
+* **REMOVEDSINCE** - _OPTIONAL_ No data. Not used anymore?
+
+### FILE
+* **NAME** - Name of file
+* **CHECKSUM** - _OPTIONAL_ SHA256 hash of file
+* **PACKAGE** - _OPTIONAL_ Package that contains this file. Not present on removed files or files that have been patched.
+* **REMOVEDSINCE** - _OPTIONAL_ Version this package was removed in. If present, only _NAME_ and _REMOVEDSINCE_ will be present.
+  * **PATCH**
+    * **FROM** - Version that the file last changed
+    * **PACKAGE** - Name of package that contains this file.

docs/file_formats/MEGAFEST.md

@@ -0,0 +1,319 @@
+> [!NOTE]
+> DRAFT
+
+# Megafest
+
+The fat manifest for W3D Hub Packager and Launcher(s), represented as a JSON hash.
+
+This **MEGAFEST** is intended to simplify patch generation and distribution by ensuring that
+all the data the Packager and Launcher(s) need can be found in a **single** manifest file.
+
+Speed up application packaging by only needing to scan and checksum the new builds files instead of needing
+to maintain a pristine copy of the previous version to be diffed against.
+
+Reduce the number of requests that the launcher(s) need to make to sort out how to download and
+patch the application.
+
+## Glossary
+
+### File
+
+An individual file on disk.
+
+An individual file as part of a MIX archive.
+
+### Package
+
+A collection of one or more files in a compressed archive for efficiently distributing the application.
+
+> [!NOTE]
+> Packages _MAY_ be compressed using ZSTANDARD (.zst) in the future. Do not assume all packages are zip files.
+>
+> Package names in the megafest include the file extension for this reason. 
+
+## Example `megafest.json` File
+
+```json
+{
+  "spec": 0,
+  "application": {
+    "id": "apb",
+    "version": "3.7.0.1",
+    "user_level": "public",
+    "previous_versions": [
+      "3.7.0.0"
+    ]
+  },
+  "dependencies": [
+    "msvc-2022"
+  ],
+  "packages": [
+    {
+      "name": "binaries.zip",
+      "version": "3.7.0.0",
+      "checksum": "SHA256-HASH",
+      "size": 4096,
+      "chunk_size": 4194304,
+      "chunk_checksums": [
+        "SHA256-HASH-A",
+        "SHA256-HASH-B"
+      ],
+      "files": [
+        {
+          "name": "game2.exe",
+          "checksum": "SHA256-HASH",
+          "size": 8691743
+        }
+      ]
+    },
+    {
+      "name": "data/Always.dat.zip",
+      "version": "3.7.0.0",
+      "checksum": "SHA256-HASH",
+      "size": 1355917483,
+      "chunk_size": 4194304,
+      "chunk_checksums": [
+        "SHA256-HASH-A",
+        "SHA256-HASH-B"
+      ],
+      "files": [
+        {
+          "name": "mp_wep_gdi.w3d",
+          "checksum": "CRC32-HASH",
+          "offset": 0,
+          "size": 8691743
+        }
+      ]
+    },
+    {
+      "name": "data/Always.patch.3.7.0.0.zst",
+      "version": "3.7.0.1",
+      "checksum": "SHA256-HASH",
+      "size": 1355919483,
+      "chunk_size": 4194304,
+      "chunk_checksums": [
+        "SHA256-HASH-A",
+        "SHA256-HASH-B"
+      ],
+      "from_version": "3.7.0.0",
+      "files": [
+        {
+          "name": "data/Always.patch",
+          "checksum": "SHA256-HASH",
+          "size": 8691743
+        }
+      ]
+    },
+    {
+      "name": "binaries.zip",
+      "version": "3.7.0.1",
+      "checksum": "SHA256-HASH",
+      "size": 4096,
+      "chunk_size": 4194304,
+      "chunk_checksums": [
+        "SHA256-HASH-A",
+        "SHA256-HASH-B"
+      ],
+      "files": [
+        {
+          "name": "game.exe",
+          "checksum": "SHA256-HASH",
+          "size": 8691743
+        }
+      ]
+    }
+  ],
+  "changes": [
+    {
+      "name": "game.exe",
+      "type": "added",
+      "package": "binaries"
+    },
+    {
+      "name": "game2.exe",
+      "type": "removed"
+    },
+    {
+      "name": "data/Always.dat",
+      "type": "updated",
+      "package": "Always.dat"
+    }
+  ],
+  "index": [
+    {
+      "name": "game.exe",
+      "checksum": "SHA256-HASH",
+      "size": 8691743
+    },
+    {
+      "name": "data/Always.dat",
+      "checksum": "SHA256-HASH",
+      "size": 1355917483
+    }
+  ]
+}
+```
+
+## SPECIFICATION
+
+### Application
+
+List of application details:
+
+* **ID** - Unique Application ID
+* **VERSION** - Application Unique Version
+* **USER_LEVEL** - User access level for this version
+* **PREVIOUS_VERSIONS** - Complete list of previous versions. Full builds only include _the_ previous version
+
+Example:
+
+```json
+{
+  "id": "apb",
+  "version": "3.7.0.1",
+  "user_level": "public",
+  "previous_versions": [
+    "3.7.0.0"
+  ]
+}
+```
+
+### Dependencies
+
+List of application's dependencies
+
+Example:
+
+```json
+[
+  "msvc-2022"
+]
+```
+
+### Packages
+
+Complete list of application's version tree of packages.
+
+That is, every package from the last full build to present for this version series.
+
+* **NAME** - Name of package
+* **VERSION** - Application version this package belongs to
+* **CHECKSUM** - SHA256 checksum of package
+* **SIZE** - File size of package in bytes
+* CHUNK **SIZE** - Size _CHUNK_CHECKSUMS_ represent in bytes
+* **CHUNK_CHECKSUMS** - Array of file chunk checksums
+* **FROM_VERSION** - *OPTIONAL* Version this file was last changed. Only present for MIX patches.
+* **FILES** - Array of files
+    * **NAME**
+    * **CHECKSUM** - SHA256 checksum of file.
+    * **SIZE** - File size in bytes
+
+Example:
+
+```json
+[
+  {
+    "name": "binaries",
+    "version": "3.7.0.0",
+    "checksum": "SHA256-HASH",
+    "size": 4096,
+    "chunk_size": 4194304,
+    "chunk_checksums": [
+      "SHA256-HASH-A",
+      "SHA256-HASH-B"
+    ],
+    "files": [
+      {
+        "name": "game.exe",
+        "checksum": "SHA256-HASH",
+        "size": 8691743
+      }
+    ]
+  },
+  {
+    "name": "data/Always.patch.3.7.0.0",
+    "version": "3.7.0.1",
+    "checksum": "SHA256-HASH",
+    "size": 2436543,
+    "chunk_size": 4194304,
+    "chunk_checksums": [
+      "SHA256-HASH-A",
+      "SHA256-HASH-B"
+    ],
+    "from_version": "3.7.0.0",
+    "files": [
+      {
+        "name": "data/Always.patch",
+        "checksum": "SHA256-HASH",
+        "size": 3636749
+      }
+    ]
+  }
+]
+```
+
+### Changes
+
+List of file changes in this build.
+
+* **NAME** - Name of file
+* **TYPE** - Type of change
+    * **ADDED** - New file has been added
+    * **UPDATED** - File has been changed
+    * **REMOVED** - File has been deleted
+* **PACKAGE** - _OPTIONAL_ Name of package, for this build version, the file can be found in
+
+```json
+[
+  {
+    "name": "game.exe",
+    "type": "updated",
+    "package": "binaries.zip"
+  },
+  {
+    "name": "game2.exe",
+    "type": "removed"
+  },
+  {
+    "name": "data/Always.dat",
+    "type": "added",
+    "package": "Always.dat.zip"
+  }
+]
+```
+
+### Index
+
+Complete list of files for this build with: filename, checksum, file size, and list of files for MIX archives.
+
+* **NAME** - File name
+* **CHECKSUM** - SHA256 hash of file
+* **SIZE** - File size
+* **FILES** - _OPTIONAL_ List of files inside of MIX archives
+  * **NAME** - File name
+  * **CHECKSUM** - CRC32 hash of file data
+  * **OFFSET** - Offset of file in MIX archive
+  * **SIZE** - Size of file data
+
+```json
+[
+  {
+    "name": "game.exe",
+    "checksum": "SHA256-HASH",
+    "size": 8691743
+  },
+  {
+    "name": "data/Always.dat",
+    "checksum": "SHA256-HASH",
+    "size": 570961432,
+    "files": [
+      {
+        "name": "mp_wep_gdi.w3d",
+        "checksum": "CRC32_HASH",
+        "offset": 0,
+        "size": 3524
+      }
+    ]
+  }
+]
+```

docs/file_formats/MIX.md

@@ -0,0 +1,132 @@
+# Westwood Renegade MIX Archive
+
+with W3D Hub / Tiberian Technologies alterations
+
+The .MIX archive file format is used by Renegade as _yes_.
+
+> [!IMPORTANT]
+> The MIX format has no notion about directories.
+>
+> Each file name **MUST** be unique.
+>
+> e.g. `data/map.dds` and `data/RA_Under/MAP.dds` have the same base file name
+> and therefore **MUST** raise an error when writing.
+
+> [!NOTE]
+> `io_pos` is the current position on the seek head of the open file
+
+## Reading
+
+* Read [Header](#header)
+    * validate MIME type (`MIX1` or `MIX2`)
+* Jump to io_pos `file_data_offset`
+* Read `file_count`
+* Read array of [`file_data`](#file-data-array)
+* Jump to io_pos `file_names_offset`
+* Read `file_count`
+* Read array of [`file_names`](#file-names-array)
+* Read file blobs
+    * For each file's [`file_data`](#file-data-array)
+        * Jump io_pos to `file_content_offset`
+        * Read `file_content_length`
+* Done.
+
+## Writing
+
+* Write MIME type (`MIX1` or `MIX2`)
+* Jump to io_pos `16`
+* Write file blobs
+    * **IMPORTANT:** Add `-io_pos & 7` padding to io_pos **AFTER** each file blob is written
+    * Save io_pos as `file_data_offset`
+* Write `file_count`
+    * Write array of [`file_data`](#file-data-array)
+    * Save io_pos as `file_names_offset`
+* Write `file_count`
+    * Write array of [`file_names`](#file-names-array)
+        * **IMPORTANT:** Ensure file names are null terminated and DO NOT exceed 254 (+ null byte) characters in length.
+* Jump to io_pos `4`
+    * Write [`file_data_offset`](#header)
+    * Write [`file_names_offset`](#header)
+    * Write [`reserved`](#header)
+* Done.
+
+## Pseudo Example:
+
+```yml
+header:
+  mime: MIX1 or MIX2
+  file_data_offset: 0x1024EAEA
+  file_names_offset: 0x8192EAEA
+  reserved: 0
+
+files:
+  array:
+    - file_blob
+
+file_data:
+  file_count: 0x000021
+  array:
+    file_name_crc32: 0xEFEFEFEF
+    file_content_offset: 0x3217DEAD
+    file_content_length: 0x0018BEEF
+
+file_names:
+  file_count: 0x000021
+  array:
+    file_name_length: 0xff
+    file_name: "mp_wep_gdi.w3d"
+```
+
+## MIME Types
+
+### MIX1 `0x3158494D`
+
+Westwood Farm Fresh Organic.
+
+### MIX2 `0x3258494D`
+
+Same as `MIX1`. `MIX2` hints to the engine's file reader to decrypt files before use.
+
+## Data Structures
+
+### Header
+
+| Name              | Offset | Type    | Description                                                         |
+|-------------------|--------|---------|---------------------------------------------------------------------|
+| MIME              | 0      | int32_t | 4 bytes representing `MIX1` (`0x3158494D)` or `MIX2` (`0x3258494D`) |
+| File Data Offset  | 4      | int32_t | Offset in MIX archive that [`file_data`](#file-data) data starts    |
+| File Names Offset | 8      | int32_t | Offset in MIX archive that [`file_name`](#file-names) data starts   |
+| RESERVED          | 12     | int32_t | Unused reserved int. Write as `0`                                   |
+
+### File
+
+| Name      | Offset  | Type                        | Description   |
+|-----------|---------|-----------------------------|---------------|
+| File Blob | complex | char[`file_content_length`] | File contents |
+
+### File Data
+
+| Name       | Offset             | Type    | Description                    |
+|------------|--------------------|---------|--------------------------------|
+| File Count | `file_data_offset` | int32_t | Number of files in MIX archive |
+
+### File Data Array
+
+| Name                | Offset                                     | Type     | Description                                            |
+|---------------------|--------------------------------------------|----------|--------------------------------------------------------|
+| File Name CRC32     | `file_data_offset` + 4 * (index * sizeof)  | uint32_t | CRC32 of **UPPERCASE** file name in network byte order |
+| File Content Offset | `file_data_offset` + 8 * (index * sizeof)  | uint32_t | File content offset                                    |
+| File Content Length | `file_data_offset` + 12 * (index * sizeof) | uint32_t | File content length                                    |
+
+### File Names
+
+| Name       | Offset              | Type    | Description                    |
+|------------|---------------------|---------|--------------------------------|
+| File Count | `file_names_offset` | int32_t | Number of files in MIX archive |
+
+### File Names Array
+
+| Name             | Offset  | Type                     | Description                                     |
+|------------------|---------|--------------------------|-------------------------------------------------|
+| File Name Length | complex | uint8_t                  | Length of string in bytes, including null byte. |
+| File Name        | complex | char[`file_name_length`] | null terminated string                          |

docs/file_formats/W3D_HUB_PATCH.md

@@ -0,0 +1,19 @@
+# W3D Hub Patch
+A W3D Hub Patch
+
+A `.w3dhub.patch` is included in MIX Patch files.
+It contains instructions for updating the target MIX.
+
+JSON file with a static CRC32 of `E6FE46B8`
+
+## Example:
+```json
+{
+  "removedFiles": [
+    "sov_v_mad.gif"
+  ],
+  "updatedFiles": [
+    "sov_v_mad.w3d"
+  ]
+}
+```

docs/protocols/LAN_DISCOVERY.md

@@ -0,0 +1,36 @@
+> [!NOTE]
+> DRAFT
+
+# LAN Discovery
+
+Doing things
+
+Broadcast port: 4898
+
+| Name              | Type     | Description                                                              |
+|-------------------|----------|--------------------------------------------------------------------------|
+| Version           | int32_t  | Version of protocol this application supports                            |
+| Owner             | string   | Nickname of application's active user                                    |
+| Hostname          | string   | Name of device                                                           |
+| UUID              | string   | Unique identifier for this application                                   |
+| Application       | string   | Name of application                                                      |
+| Features          | array    | Array of strings naming features the application supports                |
+| Service Port      | uint16_t | Dynamically assigned TCP port that interested parties should connect too |
+
+> [!NOTE]
+> Max packet size for UDP broadcast may need to be limited to 512 bytes
+
+```json
+{
+  "version": 0,
+  "owner": "cyberarm",
+  "hostname": "PC-1692",
+  "uuid": "019bcf1e-a22e-7fe0-a3db-3a9d37bfc6fa",
+  "application": "W3D Hub Linux Launcher",
+  "features": [
+    "launcher_remote:3",
+    "package_share:1"
+  ],
+  "service_port": 56802
+}
+```

docs/protocols/LAN_PACKAGE_SHARE.md

@@ -0,0 +1,9 @@
+> [!NOTE]
+> DRAFT
+
+# LAN Package Share
+
+Authentication is not required, only user opt-in to enable package sharing from their application.
+
+* List available packages
+* upload or download verified packages between other applications on LAN network

docs/protocols/LAUNCHER_REMOTE.md

@@ -0,0 +1,9 @@
+> [!NOTE]
+> DRAFT
+
+# Launcher Remote
+
+* Get list of available, installed, and in progress applications
+* Trigger download or update of application
+* Delete application
+* Join server

lib/api.rb

@@ -3,7 +3,7 @@ class W3DHub
 
     LOG_TAG = "W3DHub::Api".freeze
 
-    API_TIMEOUT = 30 # seconds
+    API_TIMEOUT = 10 # seconds
     USER_AGENT = "Cyberarm's Linux Friendly W3D Hub Launcher v#{W3DHub::VERSION}".freeze
     DEFAULT_HEADERS = [
       ["user-agent", USER_AGENT],
@@ -16,31 +16,7 @@ class W3DHub
     ].freeze
 
     def self.on_thread(method, *args, &callback)
-      BackgroundWorker.foreground_job(-> { Api.send(method, *args) }, callback)
+      Api.send(method, *args, &callback)
-    end
-
-    class Response
-      def initialize(error: nil, status: -1, body: "")
-        @status = status
-        @body = body
-        @error = error
-      end
-
-      def success?
-        @status == 200
-      end
-
-      def status
-        @status
-      end
-
-      def body
-        @body
-      end
-
-      def error
-        @error
-      end
     end
 
     #! === W3D Hub API === !#
@@ -49,7 +25,9 @@ class W3DHub
 
     HTTP_CLIENTS = {}
 
-    def self.async_http(method, path, headers = DEFAULT_HEADERS, body = nil, backend = :w3dhub)
+    def self.async_http(method:, path:, headers:, body:, backend:, async:, &callback)
+      raise "NO CALLBACK DEFINED!" unless callback
+
       case backend
       when :w3dhub
         endpoint   = W3DHUB_API_ENDPOINT
@@ -60,15 +38,15 @@ class W3DHub
       end
 
       # Handle arbitrary urls that may come through
-      url = nil
       if path.start_with?("http")
         uri = URI(path)
+
         endpoint = uri.origin
         path = uri.request_uri
-      else
-        url = "#{endpoint}#{path}"
       end
 
+      url = "#{endpoint}#{path}"
+
       logger.debug(LOG_TAG) { "Fetching #{method.to_s.upcase} \"#{url}\"..." }
 
       # Inject Authorization header if account data is populated
@@ -78,52 +56,20 @@ class W3DHub
         headers << ["authorization", "Bearer #{Store.account.access_token}"]
       end
 
-      Sync do
+      Store.network_manager.request(method, url, headers, body, async, &callback)
-        begin
-          response = provision_http_client(endpoint).send(method, path, headers, body)
-
-          Response.new(status: response.status, body: response.read)
-        rescue Async::TimeoutError => e
-          logger.error(LOG_TAG) { "Connection to \"#{url}\" timed out after: #{API_TIMEOUT} seconds" }
-
-          Response.new(error: e)
-        rescue StandardError => e
-          logger.error(LOG_TAG) { "Connection to \"#{url}\" errored:" }
-          logger.error(LOG_TAG) { e }
-
-          Response.new(error: e)
-        ensure
-          response&.close
-        end
-      end
     end
 
-    def self.provision_http_client(hostname)
+    def self.post(path:, headers: DEFAULT_HEADERS, body: nil, backend: :w3dhub, async: true, &callback)
-      # Pin http clients to their host Thread so the fiber scheduler doesn't get upset and raise an error
+      async_http(method: :post, path: path, headers: headers, body: body, backend: backend, async: async, &callback)
-      HTTP_CLIENTS[Thread.current] ||= {}
-      return HTTP_CLIENTS[Thread.current][hostname.downcase] if HTTP_CLIENTS[Thread.current][hostname.downcase]
-
-      ssl_context = W3DHub.ca_bundle_path ? OpenSSL::SSL::SSLContext.new : nil
-      ssl_context&.set_params(
-        ca_file: W3DHub.ca_bundle_path,
-        verify_mode: OpenSSL::SSL::VERIFY_PEER
-      )
-
-      endpoint = Async::HTTP::Endpoint.parse(hostname, ssl_context: ssl_context)
-      HTTP_CLIENTS[Thread.current][hostname.downcase] = Async::HTTP::Client.new(endpoint)
     end
 
-    def self.post(path, headers = DEFAULT_HEADERS, body = nil, backend = :w3dhub)
+    def self.get(path:, headers: DEFAULT_HEADERS, body: nil, backend: :w3dhub, async: true, &callback)
-      async_http(:post, path, headers, body, backend)
+      async_http(method: :get, path: path, headers: headers, body: body, backend: backend, async: async, &callback)
-    end
-
-    def self.get(path, headers = DEFAULT_HEADERS, body = nil, backend = :w3dhub)
-      async_http(:get, path, headers, body, backend)
     end
 
     # Api.get but handles any URL instead of known hosts
-    def self.fetch(path, headers = DEFAULT_HEADERS, body = nil, backend = nil)
+    def self.fetch(path:, headers: DEFAULT_HEADERS, body: nil, backend: :w3dhub, async: true, &callback)
-      async_http(:get, path, headers, body, backend)
+      async_http(method: :get, path: path, headers: headers, body: body, backend: backend, async: async, &callback)
     end
 
     # Method: POST
@@ -141,220 +87,270 @@ class W3DHub
     #
     # On a failed login the service responds with:
     # {"error":"login-failed"}
-    def self.refresh_user_login(refresh_token, backend = :w3dhub)
+    def self.refresh_user_login(refresh_token, backend = :w3dhub, &callback)
-      body = URI.encode_www_form("data": JSON.dump({refreshToken: refresh_token}))
+      body = URI.encode_www_form("data": JSON.dump({ refreshToken: refresh_token }))
-      response = post("/apis/launcher/1/user-login", FORM_ENCODED_HEADERS, body, backend)
 
-      if response.status == 200
+      handler = lambda do |result|
-        user_data = JSON.parse(response.body, symbolize_names: true)
+        if result.okay?
+          user_data = JSON.parse(result.data, symbolize_names: true)
 
-        return false if user_data[:error]
+          if user_data[:error]
+            callback.call(CyberarmEngine::Result.new(data: false))
+            next
+          end
 
-        user_details_data = user_details(user_data[:userid]) || {}
+          user_details_data = user_details(user_data[:userid]) || {}
 
-        Account.new(user_data, user_details_data)
+          callback.call(CyberarmEngine::Result.new(data: Account.new(user_data, user_details_data)))
-      else
+        else
-        logger.error(LOG_TAG) { "Failed to fetch refresh user login:" }
+          logger.error(LOG_TAG) { "Failed to fetch refresh user login:" }
-        logger.error(LOG_TAG) { response }
+          logger.error(LOG_TAG) { result.error }
-        false
+
+          callback.call(result)
+        end
       end
+
+      post(path: "/apis/launcher/1/user-login", headers: FORM_ENCODED_HEADERS, body: body, backend: backend, &handler)
     end
 
     # See #user_refresh_token
-    def self.user_login(username, password, backend = :w3dhub)
+    def self.user_login(username, password, backend = :w3dhub, &callback)
-      body = URI.encode_www_form("data": JSON.dump({username: username, password: password}))
+      body = URI.encode_www_form("data": JSON.dump({ username: username, password: password }))
-      response = post("/apis/launcher/1/user-login", FORM_ENCODED_HEADERS, body, backend)
 
-      if response.status == 200
+      handler = lambda do |result|
-        user_data = JSON.parse(response.body, symbolize_names: true)
+        if result.okay?
+          user_data = JSON.parse(result.data, symbolize_names: true)
 
-        return false if user_data[:error]
+          if user_data[:error]
+            callback.call(CyberarmEngine::Result.new(data: false))
+            next
+          end
 
-        user_details_data = user_details(user_data[:userid]) || {}
+          user_details_data = user_details(user_data[:userid]) || {}
 
-        Account.new(user_data, user_details_data)
+          callback.call(CyberarmEngine::Result.new(data: Account.new(user_data, user_details_data)))
-      else
+        else
-        logger.error(LOG_TAG) { "Failed to fetch user login:" }
+          logger.error(LOG_TAG) { "Failed to fetch user login:" }
-        logger.error(LOG_TAG) { response }
+          logger.error(LOG_TAG) { result.error }
-        false
+
+          callback.call(result)
+        end
       end
+
+      post(path: "/apis/launcher/1/user-login", headers: FORM_ENCODED_HEADERS, body: body, backend: backend, &handler)
     end
 
     # /apis/w3dhub/1/get-user-details
     #
     # Response: avatar-uri (Image download uri), id, username
-    def self.user_details(id, backend = :w3dhub)
+    def self.user_details(id, backend = :w3dhub, &callback)
       body = URI.encode_www_form("data": JSON.dump({ id: id }))
-      user_details = post("/apis/w3dhub/1/get-user-details", FORM_ENCODED_HEADERS, body, backend)
 
-      if user_details.status == 200
+      handler = lambda do |result|
-        JSON.parse(user_details.body, symbolize_names: true)
+        if result.okay?
-      else
+          callback.call(CyberarmEngine::Result.new(data: JSON.parse(result.data, symbolize_names: true)))
-        logger.error(LOG_TAG) { "Failed to fetch user details:" }
+        else
-        logger.error(LOG_TAG) { user_details }
+          logger.error(LOG_TAG) { "Failed to fetch user details:" }
-        false
+          logger.error(LOG_TAG) { result.error }
+
+          callback.call(result)
+        end
       end
+
+      post(path: "/apis/w3dhub/1/get-user-details", headers: FORM_ENCODED_HEADERS, body: body, backend: backend, &handler)
     end
 
     # /apis/w3dhub/1/get-service-status
     # Service response:
     # {"services":{"authentication":true,"packageDownload":true}}
-    def self.service_status(backend = :w3dhub)
+    def self.service_status(backend = :w3dhub, &callback)
-      response = post("/apis/w3dhub/1/get-service-status", DEFAULT_HEADERS, nil, backend)
+      handler = lambda do |result|
+        if result.okay?
+          callback.call(CyberarmEngine::Result.new(data: ServiceStatus.new(result.data)))
+        else
+          logger.error(LOG_TAG) { "Failed to fetch service status:" }
+          logger.error(LOG_TAG) { result.error }
 
-      if response.status == 200
+          callback.call(result)
-        ServiceStatus.new(response.body)
+        end
-      else
-        logger.error(LOG_TAG) { "Failed to fetch service status:" }
-        logger.error(LOG_TAG) { response }
-        false
       end
+
+      post(path: "/apis/w3dhub/1/get-service-status", backend: backend, &handler)
     end
 
     # /apis/launcher/1/get-applications
     # Client sends an Authorization header bearer token which is received from logging in (Optional)
     # Launcher sends an empty data request: data={}
     # Response is a list of applications/games
-    def self.applications(backend = :w3dhub)
+    def self.applications(backend = :w3dhub, &callback)
-      response = post("/apis/launcher/1/get-applications", DEFAULT_HEADERS, nil, backend)
+      async = !callback.nil?
 
-      if response.status == 200
+      # Complicated why to "return" direct value
-        Applications.new(response.body, backend)
+      callback = ->(result) { result }
-      else
+
-        logger.error(LOG_TAG) { "Failed to fetch applications list:" }
+      handler = lambda do |result|
-        logger.error(LOG_TAG) { response }
+        if result.okay?
-        false
+          callback.call(CyberarmEngine::Result.new(data: Applications.new(result.data, backend)))
+        else
+          logger.error(LOG_TAG) { "Failed to fetch applications list:" }
+          logger.error(LOG_TAG) { result.error }
+
+          callback.call(result)
+        end
       end
+
+      post(path: "/apis/launcher/1/get-applications", async: async, backend: backend, &handler)
     end
 
     # Populate applications list from primary and alternate backends
     # (alternate only has latest public builds of _most_ games)
-    def self._applications
+    def self._applications(&callback)
-      applications_primary = Store.account ? Api.applications(:w3dhub) : false
+      handler = lambda do |result|
-      applications_alternate = Api.applications(:alt_w3dhub)
+        # nothing special on offer if we're not logged in
+        applications_primary = Store.account ? Api.applications(:w3dhub).data : false
+        applications_alternate = Api.applications(:alt_w3dhub).data
 
-      # Fail if we fail to fetch applications list from either backend
+        # Fail if we fail to fetch applications list from either backend
-      return false unless applications_primary || applications_alternate
+        unless applications_primary || applications_alternate
+          callback.call(CyberarmEngine::Result.new)
+          next
+        end
 
-      return applications_alternate unless applications_primary
+        unless applications_primary
+          callback.call(CyberarmEngine::Result.new(data: applications_alternate))
+          next
+        end
 
-      # Merge the two app lists together
+        # Merge the two app lists together
-      apps = applications_alternate
+        apps = applications_alternate
-      if applications_primary
+        if applications_primary
-        applications_primary.games.each do |game|
+          applications_primary.games.each do |game|
-          # Check if game exists in alternate list
+            # Check if game exists in alternate list
-          _game = apps.games.find { |g| g.id == game.id }
+            _game = apps.games.find { |g| g.id == game.id }
-          unless _game
+            unless _game
-            apps.games << game
+              apps.games << game
 
-            # App didn't exist in alternates list
+              # App didn't exist in alternates list
-            # comparing channels isn't useful
+              # comparing channels isn't useful
-            next
-          end
-
-          # If it does, check that all of its channels also exist in alternate list
-          # and that the primary versions are the same as the alternates list
-          game.channels.each do |channel|
-            _channel = _game.channels.find { |c| c.id == channel.id }
-
-            unless _channel
-              _game.channels << channel
-
-              # App didn't have channel in alternates list
-              # comparing channel isn't useful
               next
             end
 
-            # If channel versions and access levels match then all's well
+            # If it does, check that all of its channels also exist in alternate list
-            if channel.current_version == _channel.current_version &&
+            # and that the primary versions are the same as the alternates list
-              channel.user_level == _channel.user_level
+            game.channels.each do |channel|
+              _channel = _game.channels.find { |c| c.id == channel.id }
 
-              # All's Well!
+              unless _channel
-              next
+                _game.channels << channel
-            end
 
-            # If the access levels don't match then overwrite alternate's channel with primary's channel
+                # App didn't have channel in alternates list
-            if channel.user_level != _channel.user_level
+                # comparing channel isn't useful
-              # Replace alternate's channel with primary's channel
+                next
-              _game.channels[_game.channels.index(_channel)] = channel
+              end
 
-              # Replaced, continue.
+              # If channel versions and access levels match then all's well
-              next
+              if channel.current_version == _channel.current_version &&
-            end
+                channel.user_level == _channel.user_level
 
-            # If versions don't match then pick whichever one is higher
+                # All's Well!
-            if Gem::Version.new(channel.current_version) > Gem::Version.new(_channel.current_version)
+                next
-              # Replace alternate's channel with primary's channel
+              end
-              _game.channels[_game.channels.index(_channel)] = channel
+
-            else
+              # If the access levels don't match then overwrite alternate's channel with primary's channel
-              # Do nothing, alternate backend version is greater.
+              if channel.user_level != _channel.user_level
+                # Replace alternate's channel with primary's channel
+                _game.channels[_game.channels.index(_channel)] = channel
+
+                # Replaced, continue.
+                next
+              end
+
+              # If versions don't match then pick whichever one is higher
+              if Gem::Version.new(channel.current_version) > Gem::Version.new(_channel.current_version)
+                # Replace alternate's channel with primary's channel
+                _game.channels[_game.channels.index(_channel)] = channel
+              else
+                # Do nothing, alternate backend version is greater.
+              end
             end
           end
         end
+
+        callback.call(CyberarmEngine::Result.new(data: apps))
       end
 
-      apps
+      # Bit hacky but we just need to run this handler from the networking thread and async reactor
+      get(path: "", backend: nil, &handler)
     end
 
     # /apis/w3dhub/1/get-news
     # Client sends an Authorization header bearer token which is received from logging in (Optional)
     # Client requests news for a specific application/game e.g.: data={"category":"ia"} ("launcher-home" retrieves the weekly hub updates)
     # Response is a JSON hash with a "highlighted" and "news" keys; the "news" one seems to be the desired one
-    def self.news(category, backend = :w3dhub)
+    def self.news(category, backend = :w3dhub, &callback)
-      body = URI.encode_www_form("data": JSON.dump({category: category}))
+      handler = lambda do |result|
-      response = post("/apis/w3dhub/1/get-news", FORM_ENCODED_HEADERS, body, backend)
+        if result.okay?
+          callback.call(CyberarmEngine::Result.new(data: News.new(result.data)))
+        else
+          logger.error(LOG_TAG) { "Failed to fetch news for:" }
+          logger.error(LOG_TAG) { category }
+          logger.error(LOG_TAG) { result.error }
 
-      if response.status == 200
+          callback.call(result)
-        News.new(response.body)
+        end
-      else
-        logger.error(LOG_TAG) { "Failed to fetch news for:" }
-        logger.error(LOG_TAG) { category }
-        logger.error(LOG_TAG) { response }
-        false
       end
+
+      body = URI.encode_www_form("data": JSON.dump({ category: category }))
+      post(path: "/apis/w3dhub/1/get-news", headers: FORM_ENCODED_HEADERS, body: body, backend: backend, &handler)
     end
 
     # Downloading games
 
     # /apis/launcher/1/get-package-details
     # client requests package details: data={"packages":[{"category":"games","name":"apb.ico","subcategory":"apb","version":""}]}
-    def self.package_details(packages, backend = :w3dhub)
+    def self.package_details(packages, backend = :w3dhub, &callback)
-      body = URI.encode_www_form("data": JSON.dump({ packages: packages }))
+      handler = lambda do |result|
-      response = post("/apis/launcher/1/get-package-details", FORM_ENCODED_HEADERS, body, backend)
+        if result.okay?
+          hash = JSON.parse(result.data, symbolize_names: true)
 
-      if response.status == 200
+          callback.call(CyberarmEngine::Result.new(data: hash[:packages].map { |pkg| Package.new(pkg) }))
-        hash = JSON.parse(response.body, symbolize_names: true)
+        else
+          logger.error(LOG_TAG) { "Failed to fetch package details for:" }
+          logger.error(LOG_TAG) { packages }
+          logger.error(LOG_TAG) { result.error }
 
-        hash[:packages].map { |pkg| Package.new(pkg) }
+          callback.call(result)
-      else
+        end
-        logger.error(LOG_TAG) { "Failed to fetch package details for:" }
-        logger.error(LOG_TAG) { packages }
-        logger.error(LOG_TAG) { response }
-        false
       end
+
+      body = URI.encode_www_form("data": JSON.dump({ packages: packages }))
+      post(path: "/apis/launcher/1/get-package-details", headers: FORM_ENCODED_HEADERS, body: body, backend: backend, &handler)
     end
 
     # /apis/launcher/1/get-package
     # client requests package: data={"category":"games","name":"ECW_Asteroids.zip","subcategory":"ecw","version":"1.0.0.0"}
     #
     # server responds with download bytes, probably supports chunked download and resume
-    def self.package(package, &block)
+    # FIXME: REFACTOR Cache.fetch_package to use HttpClient
-      Cache.fetch_package(package, block)
+    def self.package(package, &callback)
+      Cache.fetch_package(package, callback)
     end
 
     # /apis/w3dhub/1/get-events
     #
     # clients requests events: data={"serverPath":"apb"}
-    def self.events(app_id, backend = :w3dhub)
+    def self.events(app_id, backend = :w3dhub, &callback)
-      body = URI.encode_www_form("data": JSON.dump({ serverPath: app_id }))
+      handler = lambda do |result|
-      response = post("/apis/w3dhub/1/get-server-events", FORM_ENCODED_HEADERS, body, backend)
+        if result.okay?
-
+          array = JSON.parse(result.data, symbolize_names: true)
-      if response.status == 200
+          callback.call(CyberarmEngine::Result.new(data: array.map { |e| Event.new(e) }))
-        array = JSON.parse(response.body, symbolize_names: true)
+        else
-        array.map { |e| Event.new(e) }
+          callback.call(result)
-      else
+        end
-        false
       end
+
+      body = URI.encode_www_form("data": JSON.dump({ serverPath: app_id }))
+      post(path: "/apis/w3dhub/1/get-server-events", headers: FORM_ENCODED_HEADERS, body: body, backend: backend, &handler)
     end
 
     #! === Server List API === !#
@@ -380,15 +376,17 @@ class W3DHub
     #     id, name, score, kills, deaths
     #   ...players[]:
     #     nick, team (index of teams array), score, kills, deaths
-    def self.server_list(level = 1, backend = :gsh)
+    def self.server_list(level = 1, backend = :gsh, &callback)
-      response = get("/listings/getAll/v2?statusLevel=#{level}", DEFAULT_HEADERS, nil, backend)
+      handler = lambda do |result|
-
+        if result.okay?
-      if response.status == 200
+          data = JSON.parse(result.data, symbolize_names: true)
-        data = JSON.parse(response.body, symbolize_names: true)
+          callback.call(CyberarmEngine::Result.new(data: data.map { |hash| ServerListServer.new(hash) }))
-        return data.map { |hash| ServerListServer.new(hash) }
+        else
+          callback.call(result)
+        end
       end
 
-      false
+      get(path: "/listings/getAll/v2?statusLevel=#{level}", backend: backend, &handler)
     end
 
     # /listings/getStatus/v2/:id?statusLevel=#{0-2}
@@ -402,23 +400,24 @@ class W3DHub
     #     id, name, score, kills, deaths
     #   ...players[]:
     #     nick, team (index of teams array), score, kills, deaths
-    def self.server_details(id, level, backend = :gsh)
+    def self.server_details(id, level, backend = :gsh, &callback)
       return false unless id && level
 
-      response = get("/listings/getStatus/v2/#{id}?statusLevel=#{level}", DEFAULT_HEADERS, nil, backend)
+      handler = lambda do |result|
-
+        if result.okay?
-      if response.status == 200
+          callback.call(CyberarmEngine::Result.new(data: JSON.parse(result.data, symbolize_names: true)))
-        hash = JSON.parse(response.body, symbolize_names: true)
+        else
-        return hash
+          callback.call(result)
+        end
       end
 
-      false
+      get(path: "/listings/getStatus/v2/#{id}?statusLevel=#{level}", backend: backend, &handler)
     end
 
     # /listings/push/v2/negotiate?negotiateVersion=1
     ##? /listings/push/v2/?id=#{websocket token?}
     ## Websocket server list listener
-    def self.server_list_push(id)
+    def self.server_list_push(id, &callback)
     end
   end
 end

lib/api/server_list_server.rb

@@ -20,8 +20,12 @@ class W3DHub
 
         @status  = Status.new(@data[:status])
 
-        @ping_interval = 30_000
+        # if we're on unix and using the PingManager then check every second since
+        # we're not _actually_ pinging the server.
+        @ping_interval = W3DHub.unix? ? 1_000 : 60_000
         @last_pinged = Gosu.milliseconds + @ping_interval + 1_000
+
+        Store.ping_manager.add_address(@address)
       end
 
       def update(hash)
@@ -49,6 +53,16 @@ class W3DHub
         if force_ping || Gosu.milliseconds - @last_pinged >= @ping_interval
           @last_pinged = Gosu.milliseconds
 
+          if W3DHub.unix?
+            average_ping = Store.ping_manager.ping_for(@address)
+
+            @ping = average_ping.negative? ? NO_OR_BAD_PING : average_ping
+
+            States::Interface.instance&.update_server_ping(self)
+          end
+
+          return #unless W3DHub.windows?
+
           W3DHub::BackgroundWorker.foreground_parallel_job(
             lambda do
               W3DHub.command("ping #{@address} #{W3DHub.windows? ? '-n 3' : '-c 3'}") do |line|

lib/api/server_list_updater.rb

@@ -23,7 +23,7 @@ class W3DHub
         @invocation_id = 0
 
         logger.info(LOG_TAG) { "Starting emulated SignalR Server List Updater..." }
-        run
+        # run
       end
 
       def run
@@ -32,7 +32,8 @@ class W3DHub
             begin
               @auto_reconnect = true
 
-              while W3DHub::BackgroundWorker.alive?
+              # FIXME
+              while true #W3DHub::BackgroundWorker.alive?
                 connect if @auto_reconnect
                 sleep @reconnection_delay
               end
@@ -54,19 +55,31 @@ class W3DHub
         @auto_reconnect = false
 
         logger.debug(LOG_TAG) { "Requesting connection token..." }
-        response = Api.post("/listings/push/v2/negotiate?negotiateVersion=1", Api::DEFAULT_HEADERS, "", :gsh)
 
-        if response.status != 200
+        result = nil
+        Api.post("/listings/push/v2/negotiate?negotiateVersion=1", Api::DEFAULT_HEADERS, "", :gsh) do |callback_result|
+          result = callback_result
+        end
+
+        # FIXME: we've introduced ourselves to callback hell, yay!
+        while result.nil?
+          sleep 0.1
+        end
+
+        if result.error?
           @auto_reconnect = true
-          @reconnection_delay = @reconnection_delay * 2
+          @reconnection_delay *= 2
           @reconnection_delay = 60 if @reconnection_delay > 60
+
           return
         end
 
         @reconnection_delay = 1
 
-        data = JSON.parse(response.body, symbolize_names: true)
+        connect_websocket(JSON.parse(result.data, symbolize_names: true))
+      end
 
+      def connect_websocket(data)
         @invocation_id = 0 if @invocation_id > 9095
         id = data[:connectionToken]
         endpoint = "#{Api::SERVER_LIST_ENDPOINT}/listings/push/v2?id=#{id}"

lib/application_manager.rb

@@ -5,6 +5,7 @@ class W3DHub
 
     def initialize
       @tasks = [] # :installer, :importer, :repairer, :uninstaller
+      @running_applications = {}
     end
 
     def install(app_id, channel)
@@ -22,9 +23,7 @@ class W3DHub
       # unpack packages
       # install dependencies (e.g. visual C runtime)
 
-      installer = Installer.new(app_id, channel)
+      @tasks.push(Installer.new(context: task_context(app_id, channel, "version")))
-
-      @tasks.push(installer)
     end
 
     def update(app_id, channel)
@@ -32,7 +31,7 @@ class W3DHub
 
       return false unless installed?(app_id, channel)
 
-      updater = Updater.new(app_id, channel)
+      updater = Updater.new(Installer.new(context: task_context(app_id, channel, "version")))
 
       @tasks.push(updater)
     end
@@ -97,6 +96,18 @@ class W3DHub
       Process.spawn(exe)
     end
 
+    def task_context(app_id, channel, version)
+      Task::Context.new(
+        SecureRandom.hex,
+        "games",
+        app_id,
+        channel,
+        version,
+        "",
+        ""
+      )
+    end
+
     def repair(app_id, channel)
       logger.info(LOG_TAG) { "Repair Installation Request: #{app_id}-#{channel}" }
 
@@ -109,7 +120,7 @@ class W3DHub
       # unpack packages
       # install dependencies (e.g. visual C runtime) if appropriate
 
-      @tasks.push(Repairer.new(app_id, channel))
+      @tasks.push(Repairer.new(context: task_context(app_id, channel, "version")))
     end
 
     def uninstall(app_id, channel)
@@ -124,7 +135,7 @@ class W3DHub
         title: "Uninstall #{game.name}?",
         message: "Are you sure you want to uninstall #{game.name} (#{channel})?",
         accept_callback: proc {
-          @tasks.push(Uninstaller.new(app_id, channel))
+          @tasks.push(Uninstaller.new(context: task_context(app_id, channel, "version")))
         }
       )
     end
@@ -169,11 +180,17 @@ class W3DHub
     def wine_command(app_id, channel)
       return "" if W3DHub.windows?
 
-      if !Store.settings[:wine_prefix].to_s.empty?
+      "\"#{Store.settings[:wine_command]}\" "
-        "WINEPREFIX=\"#{Store.settings[:wine_prefix]}\" \"#{Store.settings[:wine_command]}\" "
+    end
-      else
+
-        "#{Store.settings[:wine_command]} "
+    def wine_enviroment_variables(app_id, channel)
-      end
+      vars = {}
+      return vars if W3DHub.windows?
+
+      vars["WINEPREFIX"] = Store.settings[:wine_prefix] unless Store.settings[:wine_prefix].to_s.empty?
+      # vars["WINEDEBUG"] = "-all" if true # TODO make this an option. wine debug interferences with pid returned from Process.spawn
+
+      vars
     end
 
     def mangohud_command(app_id, channel)
@@ -188,6 +205,13 @@ class W3DHub
       end
     end
 
+    def mangohud_enviroment_variables(app_id, channel)
+      vars = {}
+      return vars if W3DHub.windows?
+
+      vars
+    end
+
     def dxvk_command(app_id, channel)
       return "" if W3DHub.windows?
 
@@ -201,6 +225,13 @@ class W3DHub
       end
     end
 
+    def dxvk_enviroment_variables(app_id, channel)
+      vars = {}
+      return vars if W3DHub.windows?
+
+      vars
+    end
+
     def start_command(path, exe)
       if W3DHub.windows?
         "start /D \"#{path}\" /B #{exe}"
@@ -212,17 +243,33 @@ class W3DHub
     def run(app_id, channel, *args)
       if (app_data = installed?(app_id, channel))
         install_directory = app_data[:install_directory]
-        exe_path = app_id == "ecw" ? "#{install_directory}/game500.exe" : "#{install_directory}/game.exe"
+        exe_path = app_id == "ecw" ? "#{install_directory}/game500.exe" : app_data[:install_path]
         exe_path.gsub!("/", "\\") if W3DHub.windows?
         exe_path.gsub!("\\", "/") if W3DHub.unix?
 
         exe = File.basename(exe_path)
         path = File.dirname(exe_path)
 
+        env = {}
+        if W3DHub.unix?
+          env.merge!(
+            dxvk_enviroment_variables(app_id, channel),
+            mangohud_enviroment_variables(app_id, channel),
+            wine_enviroment_variables(app_id, channel)
+          )
+        end
         attempted = false
         begin
-          pid = Process.spawn("#{dxvk_command(app_id, channel)}#{mangohud_command(app_id, channel)}#{wine_command(app_id, channel)}#{attempted ? start_command(path, exe) : "\"#{exe_path}\""} -launcher #{args.join(' ')}")
+          pid = Process.spawn(
+            env,
+            "#{dxvk_command(app_id, channel)}"\
+            "#{mangohud_command(app_id, channel)}"\
+            "#{wine_command(app_id, channel)}"\
+            "#{attempted ? start_command(path, exe) : "\"#{exe_path}\""} "\
+            "-launcher #{args.join(' ')}"
+          )
           Process.detach(pid)
+          BackgroundWorker.foreground_parallel_job(-> { monitor_process(app_id, channel, pid) }, ->(result) { handle_process_result(app_id, channel, result) })
         rescue Errno::EINVAL => e
           retryable = !attempted
           attempted = true
@@ -235,13 +282,52 @@ class W3DHub
       end
     end
 
+    def monitor_process(app_id, channel, pid)
+      key = "#{app_id}-#{channel}"
+      @running_applications[key] = pid
+
+      status = Process::Status.wait(pid)
+      pp [pid, status]
+
+      @running_applications.delete(key)
+
+      status
+    end
+
+    def handle_process_result(app_id, channel, status)
+      pp [app_id, channel, status]
+
+      # Everything's fine
+      return if status.pid >= 0 && status.success?
+
+      # Everything's not fine
+      reason = status.pid.positive? ? "Crashed" : "Failed to Launch"
+      game = Store.applications.games.find { |g| g.id == app_id }
+      title = "#{reason}: #{game.name}" if game
+      title = "Application #{reason}" unless game
+
+      message = if status.pid.negative?
+                  "Command Not Found."
+                else
+                  "Application crashed."
+                end
+
+      push_state(
+        States::MessageDialog,
+        title: title,
+        message: message,
+        accept_callback: proc {
+        }
+      )
+    end
+
     def join_server(app_id, channel, server, username = Store.settings[:server_list_username], password = nil, multi = false)
-      if installed?(app_id, channel) && username.to_s.length.positive?
+      return unless installed?(app_id, channel) && username.to_s.length.positive?
-        run(
+
-          app_id, channel,
+      run(
-          "+connect #{server.address}:#{server.port} +netplayername #{username}#{password ? " +password \"#{password}\"" : ""}#{multi ? " +multi" : ""}"
+        app_id, channel,
-        )
+        "+connect #{server.address}:#{server.port} +netplayername #{username}#{password ? " +password \"#{password}\"" : ""}#{multi ? " +multi" : ""}"
-      end
+      )
     end
 
     def play_now_server(app_id, channel)
@@ -251,9 +337,14 @@ class W3DHub
 
       server_options = Store.server_list.select do |server|
         server.game == app_id &&
-        server.channel == channel &&
+          server.channel == channel &&
-        !server.status.password &&
+          !server.status.password &&
-        server.status.player_count < server.status.max_players
+          server.status.player_count < server.status.max_players
+      end
+      # sort by player count HIGH to LOW
+      # and by ping LOW to HIGH
+      server_options.sort! do |a, b|
+        [b.status.player_count, a.ping] <=> [a.status.player_count, b.ping]
       end
 
       # try to find server with lowest ping and matching version
@@ -261,7 +352,7 @@ class W3DHub
       # try to find server with lowest ping and undefined version
       found_server ||= server_options.find { |server| server.version == Api::ServerListServer::NO_OR_DEFAULT_VERSION }
 
-      found_server ? found_server : nil
+      found_server
     end
 
     def play_now(app_id, channel)
@@ -464,7 +555,7 @@ class W3DHub
     end
 
     def installing?(app_id, channel)
-      @tasks.find { |t| t.is_a?(Installer) && t.app_id == app_id && t.release_channel == channel }
+      @tasks.find { |t| t.is_a?(Installer) && t.context.app_id == app_id && t.context.channel_id == channel }
     end
 
     def updateable?(app_id, channel)
@@ -517,6 +608,54 @@ class W3DHub
       app.channels.detect { |g| g.id.to_s == channel_id.to_s }
     end
 
+    def handle_task_event(event)
+      # ONLY CALL on MAIN Ractor
+      raise "Something has gone horribly wrong!" unless Ractor.main?
+
+      pp event
+      task = @tasks.find { |t| t.context.task_id == event.task_id }
+      return unless task # FIXME: This is probably a fatal error
+
+      case event.type
+      when Task::EVENT_FAILURE
+        Store.main_thread_queue << proc do
+          window.push_state(
+            W3DHub::States::MessageDialog,
+            type: event.data[:type],
+            title: event.data[:title],
+            message: event.data[:message]
+          )
+        end
+        # FIXME: Send event to Games page to trigger refresh
+
+        States::Interface.instance&.hide_application_taskbar
+        @tasks.delete(task)
+
+      when Task::EVENT_START
+        States::Interface.instance&.show_application_taskbar
+
+      when Task::EVENT_SUCCESS
+        States::Interface.instance&.hide_application_taskbar
+        @tasks.delete(task)
+        # FIXME: Send event to Games page to trigger refresh
+      when Task::EVENT_STATUS
+        task.status = event.data
+        States::Interface.instance&.update_interface_task_status(task)
+
+      when Task::EVENT_STATUS_OPERATION
+        hash = event.data
+        operation = task.status.operations[operation[:id]]
+
+        if operation
+          operation.label = hash[:label]
+          operation.value = hash[:value]
+          operation.progress = hash[:progress]
+
+          States::Interface.instance&.update_interface_task_status(task)
+        end
+      end
+    end
+
     # No application tasks are being done
     def idle?
       !busy?
@@ -537,15 +676,37 @@ class W3DHub
       @tasks.delete_if { |t| t.state == :complete || t.state == :halted || t.state == :failed }
 
       task = @tasks.find { |t| t.state == :not_started }
-      task&.start
+
+      return unless task
+
+      # mark MAIN ractor's task as started before handing off to background ractor
+      # so that we don't start up multiple tasks at once.
+      task.start
+      on_ractor(task)
+    end
+
+    def on_ractor(task)
+      raise "Something has gone horribly wrong!!!" unless Ractor.main?
+
+      ractor = Ractor.new(task) do |t|
+        t.start
+      end
+
+      Thread.new do
+        while (message_event = ractor.take)
+          break unless message_event.is_a?(Task::MessageEvent)
+
+          Store.application_manager.handle_task_event(message_event)
+        end
+      end
     end
 
     def task?(type, app_id, channel)
       @tasks.find do |t|
         t.type == type &&
-        t.app_id == app_id &&
+          t.context.app_id == app_id &&
-        t.release_channel == channel &&
+          t.context.channel_id == channel &&
-        [ :not_started, :running, :paused ].include?(t.state)
+          [ :not_started, :running, :paused ].include?(t.state)
       end
     end
   end

lib/application_manager/status.rb

@@ -1,10 +1,10 @@
 class W3DHub
   class ApplicationManager
     class Status
-      attr_reader :application, :channel, :step, :operations, :data
+      attr_reader :application, :channel, :operations, :data
-      attr_accessor :label, :value, :progress
+      attr_accessor :label, :value, :progress, :step
 
-      def initialize(application:, channel:, label: "", value: "", progress: 0.0, step: :pending, operations: {}, &callback)
+      def initialize(application:, channel:, label: "", value: "", progress: 0.0, step: :pending, operations: {})
         @application = application
         @channel = channel
 
@@ -15,17 +15,10 @@ class W3DHub
         @step = step
         @operations = operations
 
-        @callback = callback
 
         @data = {}
       end
 
-      def step=(sym)
-        @step = sym
-        @callback&.call(self)
-        @step
-      end
-
       class Operation
         attr_accessor :label, :value, :progress
 

lib/application_manager/task.rb

@@ -746,6 +746,8 @@ class W3DHub
         end
         patch_entry = patch_mix.entries.find { |e| e.name.casecmp?(".w3dhub.patch") || e.name.casecmp?(".bhppatch") }
         patch_entry.read
+        # "remove" patch meta file from patch before copying patch data
+        patch_mix.entries.delete(patch_entry)
 
         patch_info = JSON.parse(patch_entry.blob, symbolize_names: true)
 
@@ -765,20 +767,15 @@ class W3DHub
         patch_info[:updatedFiles].each do |file|
           logger.debug(LOG_TAG) { "        #{file}" }
 
-          patch = patch_mix.entries.find { |e| e.name.casecmp?(file) }
+          patch_mix.entries.each do |entry|
-          target = target_mix.entries.find { |e| e.name.casecmp?(file) }
+            target_mix.add_entry(entry: entry, replace: true)
-
-          if target
-            target_mix.entries[target_mix.entries.index(target)] = patch
-          else
-            target_mix.entries << patch
           end
         end
 
         logger.info(LOG_TAG) { "      Writing updated #{file_path}..." } if patch_info[:updatedFiles].size.positive?
         temp_mix_path = "#{temp_path}/#{File.basename(file_path)}"
-        temp_mix = W3DHub::WWMix.new(path: temp_mix_path)
+        temp_mix = W3DHub::WWMix.new(path: temp_mix_path, encrypted: target_mix.encrypted?)
-        target_mix.entries.each { |e| temp_mix.add_entry(entry: e) }
+        target_mix.entries.each { |e| temp_mix.add_entry(entry: e, replace: true) }
         unless temp_mix.save
           raise temp_mix.error_reason
         end

lib/application_manager/tasks/installer.rb

@@ -8,9 +8,7 @@ class W3DHub
       end
 
       def execute_task
-        show_application_taskbar
+        fail_fast!
-
-        fail_fast
         return false if failed?
 
         fetch_manifests
@@ -46,9 +44,6 @@ class W3DHub
         mark_application_installed
         return false if failed?
 
-        sleep 1
-        hide_application_taskbar
-
         true
       end
     end

lib/background_worker.rb

@@ -1,166 +0,0 @@
-class W3DHub
-  class BackgroundWorker
-    LOG_TAG = "W3DHub::BackgroundWorker"
-    @@instance = nil
-    @@alive = false
-
-    def self.create
-      raise "BackgroundWorker instance already exists!" if @@instance
-      logger.info(LOG_TAG) { "Starting background job worker..." }
-
-
-      @@thread = Thread.current
-      @@alive = true
-      @@run = true
-      @@instance = self.new
-
-      @@instance.handle_jobs
-    end
-
-    def self.instance
-      @@instance
-    end
-
-    def self.run?
-      @@run
-    end
-
-    def self.alive?
-      @@alive
-    end
-
-    def self.busy?
-      instance&.busy?
-    end
-
-    def self.shutdown!
-      @@run = false
-    end
-
-    def self.kill!
-      @@thread.kill
-
-      @@instance.kill!
-    end
-
-    def self.job(job, callback, error_handler = nil, data = nil)
-      @@instance.add_job(Job.new(job: job, callback: callback, error_handler: error_handler, data: data))
-    end
-
-    def self.parallel_job(job, callback, error_handler = nil, data = nil)
-      @@instance.add_parallel_job(Job.new(job: job, callback: callback, error_handler: error_handler, data: data))
-    end
-
-    def self.foreground_job(job, callback, error_handler = nil, data = nil)
-      @@instance.add_job(Job.new(job: job, callback: callback, error_handler: error_handler, deliver_to_queue: true, data: data))
-    end
-
-    def self.foreground_parallel_job(job, callback, error_handler = nil, data = nil)
-      @@instance.add_parallel_job(Job.new(job: job, callback: callback, error_handler: error_handler, deliver_to_queue: true, data: data))
-    end
-
-    def initialize
-      @busy = false
-      @jobs = []
-
-      # Jobs which are order independent
-      @parallel_busy = false
-      @thread_pool = []
-      @parallel_jobs = []
-    end
-
-    def kill!
-      @thread_pool.each(&:kill)
-
-      logger.info(LOG_TAG) { "Forcefully killed background job worker." }
-      @@alive = false
-    end
-
-    def handle_jobs
-      8.times do |i|
-        Thread.new do
-          @thread_pool << Thread.current
-
-          while BackgroundWorker.run?
-            job = @parallel_jobs.shift
-
-            @parallel_busy = true
-
-            begin
-              job&.do
-            rescue => e
-              job&.raise_error(e)
-            end
-
-            @parallel_busy = !@parallel_jobs.empty?
-
-            sleep 0.1
-          end
-        end
-      end
-
-      Thread.new do
-        @thread_pool << Thread.current
-
-        while BackgroundWorker.run?
-          job = @jobs.shift
-
-          @busy = true
-
-          begin
-            job&.do
-          rescue => e
-            job&.raise_error(e)
-          end
-
-          @busy = !@jobs.empty?
-
-          sleep 0.1
-        end
-
-        logger.info(LOG_TAG) { "Stopped background job worker." }
-        @@alive = false
-      end
-    end
-
-    def add_job(job)
-      @jobs << job
-    end
-
-    def add_parallel_job(job)
-      @parallel_jobs << job
-    end
-
-    def busy?
-      @busy || @parallel_busy
-    end
-
-    class Job
-      def initialize(job:, callback:, error_handler: nil, deliver_to_queue: false, data: nil)
-        @job = job
-        @callback = callback
-        @error_handler = error_handler
-        @deliver_to_queue = deliver_to_queue
-        @data = data
-      end
-
-      def do
-        result = @data ? @job.call(@data) : @job.call
-        deliver(result)
-      end
-
-      def deliver(result)
-        if @deliver_to_queue
-          Store.main_thread_queue << -> { @callback.call(result) }
-        else
-          @callback.call(result)
-        end
-      end
-
-      def raise_error(error)
-        logger.error error
-        @error_handler&.call(error)
-      end
-    end
-  end
-end

lib/cache.rb

@@ -9,19 +9,17 @@ class W3DHub
     end
 
     # Fetch a generic uri
-    def self.fetch(uri:, force_fetch: false, async: true, backend: :w3dhub)
+    def self.fetch(uri:, force_fetch: false, backend: :w3dhub)
       path = path(uri)
 
       if !force_fetch && File.exist?(path)
         path
-      elsif async
-        BackgroundWorker.job(
-          -> { Api.fetch(uri, W3DHub::Api::DEFAULT_HEADERS, nil, backend) },
-          ->(response) { File.open(path, "wb") { |f| f.write response.body } if response.status == 200 }
-        )
       else
-        response = Api.fetch(uri, W3DHub::Api::DEFAULT_HEADERS, nil, backend)
+        Api.fetch(path: uri, backend: backend) do |result|
-        File.open(path, "wb") { |f| f.write response.body } if response.status == 200
+          if result.okay?
+            File.open(path, "wb") { |f| f.write result.data }
+          end
+        end
       end
     end
 
@@ -90,8 +88,6 @@ class W3DHub
           result = true
         end
 
-        binding.irb unless response
-
         if response&.status == 200 || response&.status == 206
           result = true
         else

lib/common.rb

@@ -1,4 +1,8 @@
 class W3DHub
+  PLATFORM_WINDOWS = RbConfig::CONFIG["host_os"] =~ /(mingw|mswin|windows)/i
+  PLATFORM_DARWIN = RbConfig::CONFIG["host_os"] =~ /(darwin|mac os)/i
+  PLATFORM_LINUX = RbConfig::CONFIG["host_os"] =~ /(linux|bsd|aix|solaris)/i
+
   def self.format_size(bytes)
     case bytes
     when 0..1023 # Bytes
@@ -17,15 +21,15 @@ class W3DHub
   end
 
   def self.windows?
-    RbConfig::CONFIG["host_os"] =~ /(mingw|mswin|windows)/i
+    PLATFORM_WINDOWS
   end
 
   def self.mac?
-    RbConfig::CONFIG["host_os"] =~ /(darwin|mac os)/i
+    PLATFORM_DARWIN
   end
 
   def self.linux?
-    RbConfig::CONFIG["host_os"] =~ /(linux|bsd|aix|solaris)/i
+    PLATFORM_LINUX
   end
 
   def self.unix?

lib/network_manager.rb

@@ -0,0 +1,92 @@
+class W3DHub
+  # all http(s) requests for API calls and downloading images run through here
+  class NetworkManager
+    NetworkEvent = Data.define(:context, :result)
+    Request = Struct.new(:active, :context, :async, :callback)
+    Context = Data.define(
+      :request_id,
+      :method,
+      :url,
+      :headers,
+      :body
+    )
+
+    def initialize
+      @requests = []
+      @running = true
+
+      @thread = Thread.new do
+        @http_client = HttpClient.new
+
+        Sync do
+          while @running
+            request = @requests.find { |r| !r.active }
+
+            # goto sleep for an second if there is no work to be doing
+            unless request
+              sleep 1
+              next
+            end
+
+            request.active = true
+
+            Async do |task|
+              assigned_request = request
+              result = if assigned_request.context.url.empty?
+                assigned_request.callback.call(nil)
+              else
+                @http_client.handle(task, assigned_request)
+              end
+
+              @requests.delete(assigned_request)
+
+              # callback for this is already handled!
+              unless assigned_request.context.url.empty?
+                Store.main_thread_queue << -> { assigned_request.callback.call(result) }
+              end
+            end
+          end
+        end
+      end
+    end
+
+    def request(method, url, headers, body, async, &block)
+      request_id = SecureRandom.hex
+
+      request = Request.new(
+        false,
+        Context.new(
+          request_id,
+          method,
+          url,
+          headers,
+          body
+        ),
+        async,
+        block
+      )
+
+      @requests << request
+
+
+      if async
+        request_id
+      else # Not async, process immediately.
+        raise "WTF? This should NOT happen!" unless Async::Task.current?
+
+        Sync do |task|
+          assigned_request = request
+          result = @http_client.handle(task, assigned_request)
+
+          @requests.delete(assigned_request)
+          # "return" callback "value"
+          assigned_request.callback.call(result)
+        end
+      end
+    end
+
+    def busy?
+      @requests.any?(&:active)
+    end
+  end
+end

lib/network_manager/http_client.rb

@@ -0,0 +1,57 @@
+class W3DHub
+  class NetworkManager
+    # non-blocking, http requests.
+    class HttpClient
+      def initialize
+        @http_clients = {}
+      end
+
+      def handle(task, request)
+        result = CyberarmEngine::Result.new
+        context = request.context
+
+        task.with_timeout(W3DHub::Api::API_TIMEOUT) do
+          uri = URI(context.url)
+
+          response = provision_http_client(uri.origin).send(
+            context.method,
+            uri.request_uri,
+            context.headers,
+            context.body
+          )
+
+          if response.success?
+            result.data = response.read
+          else
+            result.error = response
+          end
+        rescue Async::TimeoutError => e
+          result.error = e
+        rescue StandardError => e
+          result.error = e
+        ensure
+          response&.close
+        end
+
+        result
+      end
+
+      def provision_http_client(hostname)
+        return @http_clients[hostname.downcase] if @http_clients[hostname.downcase]
+
+        ssl_context = W3DHub.ca_bundle_path ? OpenSSL::SSL::SSLContext.new : nil
+        ssl_context&.set_params(
+          ca_file: W3DHub.ca_bundle_path,
+          verify_mode: OpenSSL::SSL::VERIFY_PEER
+        )
+
+        endpoint = Async::HTTP::Endpoint.parse(hostname, ssl_context: ssl_context)
+        @http_clients[hostname.downcase] = Async::HTTP::Client.new(endpoint)
+      end
+
+      def wrapped_error(error)
+        WrappedError.new(error.class, error.message.to_s, error.backtrace)
+      end
+    end
+  end
+end

lib/pages/community.rb

@@ -65,15 +65,7 @@ class W3DHub
             para I18n.t(:"games.fetching_news"), padding: 8
           end
 
-          BackgroundWorker.foreground_job(
+          fetch_w3dhub_news
-            -> { fetch_w3dhub_news },
-            lambda do |result|
-              if result
-                populate_w3dhub_news
-                Cache.release_net_lock(result)
-              end
-            end
-          )
         end
       end
 
@@ -89,15 +81,7 @@ class W3DHub
             title I18n.t(:"games.fetching_news"), padding: 8
           end
 
-          BackgroundWorker.foreground_job(
+          fetch_w3dhub_news
-            -> { fetch_w3dhub_news },
-            lambda do |result|
-              if result
-                populate_w3dhub_news
-                Cache.release_net_lock(result)
-              end
-            end
-          )
         end
       end
 
@@ -105,19 +89,24 @@ class W3DHub
         lock = Cache.acquire_net_lock("w3dhub_news")
         return false unless lock
 
-        news = Api.news("launcher-home")
+        Api.news("launcher-home") do |result|
-        Cache.release_net_lock("w3dhub_news") unless news
+          news = result.data
 
-        return unless news
+          Cache.release_net_lock("w3dhub_news") unless news
 
-        news.items[0..15].each do |item|
+          next false unless news
-          Cache.fetch(uri: item.image, async: false, backend: :w3dhub)
+
+          news.items[0..15].each do |item|
+            Cache.fetch(uri: item.image, backend: :w3dhub)
+          end
+
+          @w3dhub_news = news
+          @w3dhub_news_expires = Gosu.milliseconds + (60 * 60 * 1000) # 1 hour (in ms)
+
+          populate_w3dhub_news
+        ensure
+          Cache.release_net_lock("w3dhub_news")
         end
-
-        @w3dhub_news = news
-        @w3dhub_news_expires = Gosu.milliseconds + (60 * 60 * 1000) # 1 hour (in ms)
-
-        "w3dhub_news"
       end
 
       def populate_w3dhub_news
@@ -125,37 +114,7 @@ class W3DHub
 
         if (feed = @w3dhub_news)
           @wd3hub_news_container.clear do
-            # feed.items.sort_by { |i| i.timestamp }.reverse[0..9].each do |item|
+            feed.items.sort_by(&:timestamp).reverse[0..9].each do |item|
-            #   flow(width: 0.5, max_width: 312, height: 128, margin: 4) do
-            #     # background 0x88_000000
-
-            #     path = Cache.path(item.image)
-
-            #     if File.exist?(path)
-            #       image path, height: 1.0, padding: 4
-            #     else
-            #       image BLACK_IMAGE, height: 1.0, padding: 4
-            #     end
-
-            #     stack(width: 0.6, height: 1.0) do
-            #       stack(width: 1.0, height: 112) do
-            #         link "<b>#{item.title}</b>", text_size: 22 do
-            #           W3DHub.url(item.uri)
-            #         end
-            #         para item.blurb.gsub(/\n+/, "\n").strip[0..180]
-            #       end
-
-            #       flow(width: 1.0) do
-            #         para item.timestamp.strftime("%Y-%m-%d"), width: 0.499
-            #         link I18n.t(:"games.read_more"), width: 0.5, text_align: :right, text_size: 22 do
-            #           W3DHub.url(item.uri)
-            #         end
-            #       end
-            #     end
-            #   end
-            # end
-
-            feed.items.sort_by { |i| i.timestamp }.reverse[0..9].each do |item|
               image_path = Cache.path(item.image)
 
               flow(width: 1.0, max_width: 1230, height: 200, margin: 8, border_thickness: 1, border_color: lighten(Gosu::Color.new(0xff_252525))) do

lib/pages/games.rb

@@ -31,30 +31,16 @@ class W3DHub
       def update
         super
 
-
         @game_news.each do |key, value|
-          next if key.end_with?("_expires")
+          next unless key.end_with?("_expires")
 
-          if Gosu.milliseconds >= @game_news["#{key}_expires"]
+          next unless Gosu.milliseconds >= value
-            @game_news.delete(key)
-            @game_news["#{key}_expires"] = Gosu.milliseconds + 30_000 # seconds
 
-            if @focused_game && @focused_game.id == key
+          # try to refresh game news after last data 'expired', every 30 seconds until success
-              @game_news_container.clear do
+          @game_news[key] = Gosu.milliseconds + 30_000 # seconds
-                title I18n.t(:"games.fetching_news"), padding: 8
-              end
 
-              BackgroundWorker.foreground_job(
+          game = Store.applications.games.find { |g| g.id == key.split("_").first }
-                -> { fetch_game_news(@focused_game) },
+          fetch_game_news(game)
-                lambda do |result|
-                  if result
-                    populate_game_news(@focused_game)
-                    Cache.release_net_lock(result)
-                  end
-                end
-              )
-            end
-          end
         end
       end
 
@@ -292,22 +278,6 @@ class W3DHub
         return if Store.offline_mode
 
         unless Cache.net_lock?("game_news_#{game.id}")
-          if @game_events[game.id]
-            populate_game_events(game)
-          else
-            BackgroundWorker.foreground_job(
-              -> { fetch_game_events(game) },
-              lambda do |result|
-                if result
-                  populate_game_events(game)
-                  Cache.release_net_lock(result)
-                end
-              end
-            )
-          end
-        end
-
-        unless Cache.net_lock?("game_events_#{game.id}")
           if @game_news[game.id]
             populate_game_news(game)
           else
@@ -315,15 +285,15 @@ class W3DHub
               title I18n.t(:"games.fetching_news"), padding: 8
             end
 
-            BackgroundWorker.foreground_job(
+            fetch_game_news(game)
-              -> { fetch_game_news(game) },
+          end
-              lambda do |result|
+        end
-                if result
+
-                  populate_game_news(game)
+        unless Cache.net_lock?("game_events_#{game.id}")
-                  Cache.release_net_lock(result)
+          if @game_events[game.id]
-                end
+            populate_game_events(game)
-              end
+          else
-            )
+            fetch_game_events(game)
           end
         end
       end
@@ -413,19 +383,25 @@ class W3DHub
         lock = Cache.acquire_net_lock("game_news_#{game.id}")
         return false unless lock
 
-        news = Api.news(game.id)
+        Api.news(game.id) do |result|
-        Cache.release_net_lock("game_news_#{game.id}") unless news
+          news = result.data
 
-        return false unless news
+          unless news
+            @game_news["#{game.id}_expires"] = Gosu.milliseconds + 30_000 # retry in 30 seconds
+            next false
+          end
 
-        news.items[0..15].each do |item|
+          news.items[0..15].each do |item|
-          Cache.fetch(uri: item.image, async: false, backend: :w3dhub)
+            Cache.fetch(uri: item.image, backend: :w3dhub)
+          end
+
+          @game_news[game.id] = news
+          @game_news["#{game.id}_expires"] = Gosu.milliseconds + (60 * 60 * 1000) # 1 hour (in ms)
+
+          populate_game_news(@focused_game)
+        ensure
+          Cache.release_net_lock("game_news_#{game.id}")
         end
-
-        @game_news[game.id] = news
-        @game_news["#{game.id}_expires"] = Gosu.milliseconds + (60 * 60 * 1000) # 1 hour (in ms)
-
-        "game_news_#{game.id}"
       end
 
       def populate_game_news(game)
@@ -436,38 +412,11 @@ class W3DHub
           game_color.alpha = 0xaa
 
           @game_news_container.clear do
-            # Patch Notes
+            feed.items.sort_by(&:timestamp).reverse[0..9].each do |item|
-            if false # Patch notes
-              flow(width: 1.0, max_width: 346 * 3 + (8 * 4), height: 346, margin: 8, margin_right: 32, border_thickness: 1, border_color: darken(Gosu::Color.new(game.color))) do
-                background darken(Gosu::Color.new(game.color), 10)
-
-                stack(width: 346, height: 1.0, padding: 8) do
-                  background 0xff_181d22
-
-                  para "Patch Notes"
-
-                  tagline "<b>Patch 2.0 is now out!</b>"
-
-                  para "words go here " * 20
-
-                  flow(fill: true)
-
-                  button "Read More", width: 1.0
-                end
-
-                flow(fill: true)
-
-                title "Eye Candy Banner Goes Here."
-              end
-            end
-
-            feed.items.sort_by { |i| i.timestamp }.reverse[0..9].each do |item|
               image_path = Cache.path(item.image)
 
               flow(width: 1.0, max_width: 869, height: 200, margin: 8, background: game_color, border_thickness: 1, border_color: lighten(Gosu::Color.new(game.color))) do
-                if File.file?(image_path)
+                image image_path, height: 1.0 if File.file?(image_path)
-                  image image_path, height: 1.0
-                end
 
                 stack(fill: true, height: 1.0, padding: 4, border_thickness_left: 1, border_color_left: lighten(Gosu::Color.new(game.color))) do
                   tagline "<b>#{item.title}</b>", width: 1.0
@@ -494,14 +443,14 @@ class W3DHub
         lock = Cache.acquire_net_lock("game_events_#{game.id}")
         return false unless lock
 
-        events = Api.events(game.id)
+        Api.events(game.id) do |result|
-        Cache.release_net_lock("game_events_#{game.id}") unless events
+          next unless result.okay?
 
-        return false unless events
+          @game_events[game.id] = result.data
-
+          populate_game_events(game)
-        @game_events[game.id] = events
+        ensure
-
+          Cache.release_net_lock("game_events_#{game.id}")
-        "game_events_#{game.id}"
+        end
       end
 
       def populate_game_events(game)

lib/pages/login.rb

@@ -34,9 +34,8 @@ class W3DHub
 
                   # Do network stuff
 
-                  BackgroundWorker.foreground_job(
+                  Api.user_login(@username.value, @password.value) do |result|
-                    lambda do
+                    if result.okay?
-                      account = Api.user_login(@username.value, @password.value)
                       applications = nil
 
                       if account
@@ -45,14 +44,15 @@ class W3DHub
                         Store.settings.save_settings
 
                         if account
-                          Cache.fetch(uri: account.avatar_uri, force_fetch: true, async: false, backend: :w3dhub)
+                          Cache.fetch(uri: account.avatar_uri, force_fetch: true, backend: :w3dhub) {}
-                          applications = Api._applications
+                          Api._applications do |r|
+                            applications = r.result if r.okay?
+                          end
                         end
                       end
 
                       [account, applications]
-                    end,
+                    else
-                    lambda do |result|
                       account, applications = result
 
                       if account
@@ -70,7 +70,7 @@ class W3DHub
                         @error_label.value = "Incorrect username or password.\nOr too many failed login attempts, try again in a few minutes."
                       end
                     end
-                  )
+                  end
                 end
 
                 @error_label = caption "", width: 1.0, text_align: :center, color: 0xff_800000
@@ -80,13 +80,10 @@ class W3DHub
         end
 
         if Store.account
-          BackgroundWorker.foreground_job(
+          Cache.fetch(uri: Store.account.avatar_uri, backend: :w3dhub) do |result|
-            -> { Cache.fetch(uri: Store.account.avatar_uri, async: false, backend: :w3dhub) },
+            populate_account_info
-            ->(result) {
+            page(W3DHub::Pages::Games)
-              populate_account_info
+          end
-              page(W3DHub::Pages::Games)
-            }
-          )
         end
       end
 
@@ -153,29 +150,26 @@ class W3DHub
         Store.settings.save_settings
         Store.account = nil
 
-        BackgroundWorker.foreground_job(
+        Api._applications do |result|
-          -> { Api._applications },
+          if result.okay?
-          lambda do |applications|
+            Store.applications = result.data
-            if applications
+            page(W3DHub::Pages::Games) if @host.current_page.is_a?(W3DHub::Pages::Games)
-              Store.applications = applications
+            page(W3DHub::Pages::ServerBrowser) if @host.current_page.is_a?(W3DHub::Pages::ServerBrowser)
-              page(W3DHub::Pages::Games) if @host.current_page.is_a?(W3DHub::Pages::Games)
+          end
-              page(W3DHub::Pages::ServerBrowser) if @host.current_page.is_a?(W3DHub::Pages::ServerBrowser)
-            end
 
-            @host.instance_variable_get(:"@account_container").clear do
+          @host.instance_variable_get(:"@account_container").clear do
-              stack(width: 1.0, height: 1.0) do
+            stack(width: 1.0, height: 1.0) do
-                tagline "<b>#{I18n.t(:"interface.not_logged_in")}</b>", text_wrap: :none
+              tagline "<b>#{I18n.t(:"interface.not_logged_in")}</b>", text_wrap: :none
 
-                flow(width: 1.0) do
+              flow(width: 1.0) do
-                  link(I18n.t(:"interface.log_in"), text_size: 22, width: 0.5) { page(W3DHub::Pages::Login) }
+                link(I18n.t(:"interface.log_in"), text_size: 22, width: 0.5) { page(W3DHub::Pages::Login) }
-                  link I18n.t(:"interface.register"), text_size: 22, width: 0.49 do
+                link I18n.t(:"interface.register"), text_size: 22, width: 0.49 do
-                    W3DHub.url("https://secure.w3dhub.com/forum/index.php?app=core&module=global&section=register")
+                  W3DHub.url("https://secure.w3dhub.com/forum/index.php?app=core&module=global&section=register")
-                  end
                 end
               end
             end
           end
-        )
+        end
       end
     end
   end

lib/pages/server_browser.rb

@@ -145,15 +145,7 @@ class W3DHub
           reorder_server_list
 
           if @selected_server&.id == @refresh_server&.id
-            if @refresh_server
+            fetch_server_details(@refresh_server) if @refresh_server
-              BackgroundWorker.foreground_job(
-                -> { fetch_server_details(@refresh_server) },
-                ->(result) {
-                  populate_server_info(@refresh_server) if @refresh_server == @selected_server
-                  @refresh_server = nil
-                }
-              )
-            end
           end
         end
       end
@@ -196,13 +188,13 @@ class W3DHub
 
       def ping_icon(server)
         case server.ping
-        when 0..150
+        when 0..50
           @ping_icons[:good]
-        when 151..200
+        when 51..150
           @ping_icons[:fair]
-        when 201..1_000
+        when 151..200
           @ping_icons[:poor]
-        when 1_001..5_000
+        when 201..1_000
           @ping_icons[:bad]
         else
           @ping_icons[:unknown]
@@ -353,10 +345,7 @@ class W3DHub
 
           reorder_server_list if @selected_server_container
 
-          BackgroundWorker.foreground_job(
+          fetch_server_details(server)
-            -> { fetch_server_details(server) },
-            ->(result) { populate_server_info(server) if server == @selected_server }
-          )
         end
 
         stylize_selected_server(server_container) if server.id == @selected_server&.id
@@ -523,10 +512,13 @@ class W3DHub
       end
 
       def fetch_server_details(server)
-        BackgroundWorker.foreground_job(
+        Api.server_details(server.id, 2) do |result|
-          -> { Api.server_details(server.id, 2) },
+          if result.okay?
-          ->(server_data) { server.update(server_data) if server_data }
+            server.update(result.data)
-        )
+            populate_server_info(server) if server == @selected_server
+            @refresh_server = nil
+          end
+        end
       end
 
       def game_icon(server)

lib/ping.rb

@@ -0,0 +1,166 @@
+require "async"
+require "socket"
+require "securerandom"
+
+class W3DHub
+  class Ping
+    ICMPHeader = Data.define(:type, :code, :checksum, :_ping_id, :_sequence_id, :data)
+    EchoRequest = Struct.new(:ping_id, :sequence_id, :data, :time, :timed_out)
+
+    ICMP_ECHOREPLY = 0
+    ICMP_ECHO      = 8
+    ICMP_SUBCODE   = 0
+
+    BIT_PACKER = "C2 n3 A*".freeze
+    MINIMUM_INTERVAL = 250 # ms # intervals below 200ms are considered rude and may be dropped due to flooding.
+    ECHO_REQUEST_HISTORY = 30 # 100 # keep the last n requests
+
+    attr_reader :address
+
+    def initialize(address:, count: 10, ttl: 120, interval: 1_000, data: nil)
+      @address = address
+      @count = count
+      @ttl = ttl
+      @interval = interval.to_i < MINIMUM_INTERVAL ? MINIMUM_INTERVAL : interval # ms
+      @data = data
+
+      # circular buffer
+      @echo_requests = Array.new(ECHO_REQUEST_HISTORY) { EchoRequest.new(-1, -1, "", nil, false) }
+      @echo_requests_index = 0
+
+      # NOTE: The PING_ID _might_ be overruled by the kernel and should not be used
+      #       to check that any received echo replies are ours.
+      #
+      #       Sequence ID and Data appear to be unmodified.
+      @ping_id = SecureRandom.hex.to_i(16) & 0xffff
+      @sequence_id = SecureRandom.hex.to_i(16) & 0xffff
+
+      addresses = Addrinfo.getaddrinfo(@address, nil, Socket::AF_INET, :DGRAM)
+      raise "NO ADDRESSES!" if addresses.empty?
+
+      @socket_address = addresses.sample.to_sockaddr
+
+      @socket = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, Socket::IPPROTO_ICMP)
+      @socket.setsockopt(Socket::SOL_SOCKET, Socket::IP_TTL, @ttl)
+    end
+
+    # Perform a checksum on the message.  This is the sum of all the short
+    # words and it folds the high order bits into the low order bits.
+    def message_checksum(message)
+      length    = message.length
+      num_short = length / 2
+      check     = 0
+
+      message.unpack("n#{num_short}").each do |short|
+        check += short
+      end
+
+      check += message[length - 1, 1].unpack1("C") << 8 if (length % 2).positive?
+
+      check = (check >> 16) + (check & 0xffff)
+      ~((check >> 16) + check) & 0xffff
+    end
+
+    def random_data
+      SecureRandom.hex
+    end
+
+    def monotonic_time
+      Process.clock_gettime(:CLOCK_MONOTONIC, :millisecond)
+    end
+
+    def verified?(message)
+      data = message.unpack(BIT_PACKER)
+      checksum = data[2]
+
+      # set checksum in message to 0
+      data[2] = 0
+
+      checksum == message_checksum(data.pack(BIT_PACKER))
+    end
+
+    def request_complete?(request)
+      request.timed_out || !request.time.nil?
+    end
+
+    def packet_loss
+      completed_requests = @echo_requests.select { |r| request_complete?(r) }
+      failed_requests = completed_requests.select(&:timed_out)
+
+      # 0% packet loss 😎
+      return 0.0 if failed_requests.empty?
+
+      # 100% packet loss
+      return 1.0 if failed_requests.size == completed_requests.size
+
+      failed_requests.size / completed_requests.size.to_f
+    end
+
+    def average_ping
+      times = @echo_requests.select { |r| request_complete?(r) && !r.timed_out }.map(&:time)
+
+      return -1 unless times.size.positive?
+
+      times.sum.to_f / times.size
+    end
+
+    # returns true if any echo requests have completed (reply received or timed out) and packet loss is less than 30%
+    def okay?
+      completed_requests = @echo_requests.select { |r| request_complete?(r) }.size
+
+      completed_requests.positive? && packet_loss < 0.3
+    end
+
+    def ping(count = @count)
+      return if count <= 0
+
+      Async do |task|
+        @count.times do
+          task.Async do |subtask|
+            @sequence_id = (@sequence_id + 1) % 0xffff
+            data = @data || random_data
+
+            checksum = 0
+            message = [ICMP_ECHO, ICMP_SUBCODE, checksum, @ping_id, @sequence_id, data].pack(BIT_PACKER)
+            checksum = message_checksum(message)
+            message = [ICMP_ECHO, ICMP_SUBCODE, checksum, @ping_id, @sequence_id, data].pack(BIT_PACKER)
+
+            @socket.send(message, 0, @socket_address)
+
+            s = monotonic_time
+            request = @echo_requests[@echo_requests_index]
+            request.ping_id = @ping_id
+            request.sequence_id = @sequence_id
+            request.data = data
+            request.time = nil
+            request.timed_out = false
+            @echo_requests_index = (@echo_requests_index + 1) % ECHO_REQUEST_HISTORY
+
+            subtask.with_timeout(2) do
+              loop do
+                data, _addrinfo = @socket.recvfrom(1500)
+
+                # ignore corruption
+                next unless verified?(data)
+
+                header = ICMPHeader.new(*data.unpack(BIT_PACKER))
+
+                if header.type == ICMP_ECHOREPLY && header._sequence_id == request.sequence_id && header.data == request.data
+                  duration = monotonic_time - s
+                  request.time = duration
+
+                  break
+                end
+              end
+            rescue Async::TimeoutError
+              request.timed_out = true
+            end
+          end
+
+          # Don't send out pings in a flood, it's considered rude.
+          sleep @interval / 1000.0
+        end
+      end
+    end
+  end
+end

lib/ping_manager.rb

@@ -0,0 +1,54 @@
+class W3DHub
+  class PingManager
+    Container = Struct.new(:pinger, :last_ping_time_ms)
+    PING_INTERVAL = 60_000
+
+    def initialize
+      @containers = {}
+      @addresses = []
+    end
+
+    def monitor(task)
+      task.async do |subtask|
+        while BackgroundWorker.alive?
+          # activate new addresses
+          @addresses.each do |address|
+            @containers[address] ||= Container.new(Ping.new(address: address), -PING_INTERVAL * 2)
+          end
+
+          # cleanup old addresses
+          @containers.each_key do |key|
+            @containers.delete(key) unless @addresses.find { |a| a == key }
+          end
+
+          # ping the pingers
+          @containers.each_value do |container|
+            next unless Gosu.milliseconds - container.last_ping_time_ms >= PING_INTERVAL
+
+            container.last_ping_time_ms = Gosu.milliseconds
+
+            subtask.async do
+              container.pinger.ping
+              # pp [container.pinger.address, container.pinger.average_ping]
+            end
+          end
+
+          sleep 0.001
+        end
+      end
+    end
+
+    def add_address(address)
+      @addresses << address
+      @addresses.uniq!
+    end
+
+    def ping_for(address)
+      @containers[address]&.pinger&.average_ping&.round || -1
+    end
+
+    def remove_address(address)
+      @addresses.delete(address)
+    end
+  end
+end

lib/states/boot.rb

@@ -115,9 +115,8 @@ class W3DHub
             Api.on_thread(:refresh_user_login, account.refresh_token) do |refreshed_account|
               update_account_data(refreshed_account)
             end
-
           else
-            BackgroundWorker.foreground_job(-> { update_account_data(account) }, ->(_) {})
+            Store.main_thread_queue << -> { update_account_data(account) }
           end
 
         else
@@ -131,7 +130,7 @@ class W3DHub
 
           Store.settings[:account][:data] = account
 
-          Cache.fetch(uri: account.avatar_uri, force_fetch: true, async: false, backend: :w3dhub)
+          Cache.fetch(uri: account.avatar_uri, force_fetch: true, backend: :w3dhub)
         else
           Store.settings[:account] = {}
         end
@@ -175,8 +174,10 @@ class W3DHub
       end
 
       def service_status
-        Api.on_thread(:service_status) do |service_status|
+        @status_label.value = "Checking service status..." #I18n.t(:"server_browser.fetching_server_list")
-          @service_status = service_status
+
+        Api.on_thread(:service_status) do |result|
+          @service_status = result.okay? ? result.data : nil
 
           if @service_status
             Store.service_status = @service_status
@@ -187,9 +188,7 @@ class W3DHub
 
             @tasks[:service_status][:complete] = true
           else
-            BackgroundWorker.foreground_job(-> {}, lambda { |_|
+            Store.main_thread_queue << -> { @status_label.value = I18n.t(:"boot.w3dhub_service_is_down") }
-              @status_label.value = I18n.t(:"boot.w3dhub_service_is_down")
-            })
             @tasks[:service_status][:complete] = true
 
             @offline_mode = true
@@ -201,9 +200,9 @@ class W3DHub
       def launcher_updater
         @status_label.value = "Checking for Launcher updates..." # I18n.t(:"boot.checking_for_updates")
 
-        Api.on_thread(:fetch, "https://api.github.com/repos/cyberarm/w3d_hub_linux_launcher/releases/latest") do |response|
+        Api.on_thread(:fetch, "https://api.github.com/repos/cyberarm/w3d_hub_linux_launcher/releases/latest") do |result|
-          if response.status == 200
+          if result.okay?
-            hash = JSON.parse(response.body, symbolize_names: true)
+            hash = JSON.parse(result.data, symbolize_names: true)
             available_version = hash[:tag_name].downcase.sub("v", "")
 
             pp Gem::Version.new(available_version) > Gem::Version.new(W3DHub::VERSION)
@@ -227,14 +226,13 @@ class W3DHub
       def applications
         @status_label.value = I18n.t(:"boot.checking_for_updates")
 
-        Api.on_thread(:_applications) do |applications|
+        Api.on_thread(:_applications) do |result|
-          if applications
+          if result.okay?
-            Store.applications = applications
+            Store.applications = result.data
-            Store.settings.save_application_cache(applications.data.to_json)
+            Store.settings.save_application_cache(Store.applications.data.to_json)
             @tasks[:applications][:complete] = true
           else
-            # FIXME: Failed to retreive!
+            @status_label.value = "FAILED TO RETRIEVE APPS LIST"
-            BackgroundWorker.foreground_job(-> {}, ->(_) { @status_label.value = "FAILED TO RETREIVE APPS LIST" })
 
             @offline_mode = true
             Store.offline_mode = true
@@ -253,40 +251,39 @@ class W3DHub
           packages << { category: app.category, subcategory: app.id, name: "#{app.id}.ico", version: "" }
         end
 
-        Api.on_thread(:package_details, packages, :alt_w3dhub) do |package_details|
+        Api.on_thread(:package_details, packages, :alt_w3dhub) do |result|
-          package_details ||= nil
+          if result.okay?
+            result.data.each do |package|
+              next if package.error?
 
-          package_details&.each do |package|
+              path = Cache.package_path(package.category, package.subcategory, package.name, package.version)
-            next if package.error?
+              generated_icon_path = "#{CACHE_PATH}/#{package.subcategory}.png"
 
-            path = Cache.package_path(package.category, package.subcategory, package.name, package.version)
+              regenerate = false
-            generated_icon_path = "#{CACHE_PATH}/#{package.subcategory}.png"
 
-            regenerate = false
+              if File.exist?(path)
-
+                broken_or_out_dated_icon = Digest::SHA256.new.hexdigest(File.binread(path)).upcase != package.checksum.upcase
-            if File.exist?(path)
-              broken_or_out_dated_icon = Digest::SHA256.new.hexdigest(File.binread(path)).upcase != package.checksum.upcase
-            end
-
-            if File.exist?(path) && !broken_or_out_dated_icon
-              regenerate = !File.exist?(generated_icon_path)
-            else
-              begin
-                Cache.fetch_package(package, proc {})
-                regenerate = true
-              rescue Errno::EACCES => e
-                failure = true
-                push_state(MessageDialog, title: "Fatal Error",
-                                          message: "Directory Permission Error (#{e.class}):\n#{e}.\n\nIs the required drive mounted?",
-                                          accept_callback: -> { window.close })
               end
+
+              if File.exist?(path) && !broken_or_out_dated_icon
+                regenerate = !File.exist?(generated_icon_path)
+              else
+                begin
+                  Cache.fetch_package(package, proc {})
+                  regenerate = true
+                rescue Errno::EACCES => e
+                  failure = true
+                  push_state(MessageDialog, title: "Fatal Error",
+                                            message: "Directory Permission Error (#{e.class}):\n#{e}.\n\nIs the required drive mounted?",
+                                            accept_callback: -> { window.close })
+                end
+              end
+
+              next unless regenerate
+
+              icon = ICO.new(file: path)
+              icon.save(icon.images.max_by(&:width), generated_icon_path)
             end
-
-            next unless regenerate
-
-            BackgroundWorker.foreground_job(-> { ICO.new(file: path) }, lambda { |result|
-              result.save(result.images.max_by(&:width), generated_icon_path)
-            })
           end
 
           @tasks[:app_icons][:complete] = true unless failure
@@ -304,18 +301,18 @@ class W3DHub
           packages << { category: app.category, subcategory: app.id, name: "background.png", version: "" }
         end
 
-        Api.on_thread(:package_details, packages, :alt_w3dhub) do |package_details|
+        Api.on_thread(:package_details, packages, :alt_w3dhub) do |result|
-          package_details ||= nil
+          if result.okay?
+            result.data.each do |package|
+              next if package.error?
 
-          package_details&.each do |package|
+              package_cache_path = Cache.package_path(package.category, package.subcategory, package.name,
-            next if package.error?
+                                                      package.version)
 
-            package_cache_path = Cache.package_path(package.category, package.subcategory, package.name,
+              missing_or_broken_image = File.exist?(package_cache_path) ? Digest::SHA256.new.hexdigest(File.binread(package_cache_path)).upcase != package.checksum.upcase : true
-                                                    package.version)
 
-            missing_or_broken_image = File.exist?(package_cache_path) ? Digest::SHA256.new.hexdigest(File.binread(package_cache_path)).upcase != package.checksum.upcase : true
+              Cache.fetch_package(package, proc {}) if missing_or_broken_image
-
+            end
-            Cache.fetch_package(package, proc {}) if missing_or_broken_image
           end
 
           @tasks[:app_logos_and_backgrounds][:complete] = true
@@ -325,15 +322,15 @@ class W3DHub
       def server_list
         @status_label.value = I18n.t(:"server_browser.fetching_server_list")
 
-        Api.on_thread(:server_list, 2) do |list|
+        Api.on_thread(:server_list, 2) do |result|
-          if list
+          if result.okay?
-            Store.server_list = list.sort_by! { |s| s&.status&.players&.size }.reverse
+            Store.server_list = result.data.sort_by! { |s| s&.status&.players&.size }.reverse
 
             Store.server_list_last_fetch = Gosu.milliseconds
 
             Api::ServerListUpdater.instance
 
-            list.each do |server|
+            Store.server_list.each do |server|
               server.send_ping(true)
             end
           else

lib/states/interface.rb

@@ -164,15 +164,15 @@ class W3DHub
         if Gosu.milliseconds >= @server_list_expire
           @server_list_expire = Gosu.milliseconds + 30_000
 
-          Api.on_thread(:server_list, 2) do |list|
+          Api.on_thread(:server_list, 2) do |result|
-            if list
+            if result.okay?
               @server_list_expire = Gosu.milliseconds + SERVER_LIST_UPDATE_INTERVAL # five minutes
 
               Store.server_list_last_fetch = Gosu.milliseconds
 
-              Api::ServerListUpdater.instance.refresh_server_list(list)
+              Api::ServerListUpdater.instance.refresh_server_list(result.data)
 
-              BackgroundWorker.foreground_job(-> {}, ->(_) { States::Interface.instance&.update_server_browser(nil, :refresh_all) })
+              Store.main_thread_queue << -> { States::Interface.instance&.update_server_browser(nil, :refresh_all) }
             end
           end
         end

lib/version.rb

@@ -1,4 +1,4 @@
 class W3DHub
   DIR_NAME = "W3DHubAlt".freeze
-  VERSION = "0.9.0".freeze
+  VERSION = "0.9.1".freeze
 end

lib/window.rb

@@ -6,7 +6,12 @@ class W3DHub
 
       Store[:server_list] = []
       Store[:settings] = Settings.new
+      Store[:network_manager] = NetworkManager.new
       Store[:application_manager] = ApplicationManager.new
+      Store[:ping_manager] = PingManager.new
+
+      # FIXME
+      # BackgroundWorker.parallel_job(-> { Async { |task| Store.ping_manager.monitor(task) } }, nil)
 
       Store[:main_thread_queue] = []
 
@@ -32,8 +37,7 @@ class W3DHub
         block&.call
       end
 
-      # Manually sleep main thread so that the BackgroundWorker thread can be scheduled
+      sleep(update_interval / 1000.0) if Store.application_manager.busy? || Store.network_manager.busy?
-      sleep(update_interval / 1000.0) if W3DHub::BackgroundWorker.busy? || Store.application_manager.busy?
     end
 
     def needs_redraw?

lib/ww_mix.rb

@@ -228,27 +228,34 @@ class W3DHub
       @encrypted
     end
 
-    def add_file(path:)
+    def add_file(path:, replace: false)
       return false unless File.exist?(path)
       return false if File.directory?(path)
 
-      info = EntryInfoHeader.new(0, 0, File.size(path))
+      entry = Entry.new(name: File.basename(path), path: path, info: EntryInfoHeader.new(0, 0, File.size(path)))
-      @entries << Entry.new(name: File.basename(path), path: path, info: info)
+      add_entry(entry: entry, replace: replace)
-
-      true
     end
 
-    def add_blob(path:, blob:)
+    def add_blob(path:, blob:, replace: false)
       info = EntryInfoHeader.new(0, 0, blob.size)
-      @entries << Entry.new(name: File.basename(path), path: path, info: info, blob: blob)
+      entry = Entry.new(name: File.basename(path), path: path, info: info, blob: blob)
       into.crc32 = @entries.last.calculate_crc32
 
-      true
+      add_entry(entry: entry, replace: replace)
     end
 
-    def add_entry(entry:)
+    def add_entry(entry:, replace: false)
-      @entries << entry
+      duplicate = @entries.find { |e| e.name.upcase == entry.name.upcase }
 
+      if duplicate
+        if replace
+          @entries.delete(duplicate)
+        else
+          return false
+        end
+      end
+
+      @entries << entry
       true
     end
 

w3d_hub_linux_launcher.rb

@@ -106,11 +106,14 @@ require_relative "lib/cache"
 require_relative "lib/settings"
 require_relative "lib/ww_mix"
 require_relative "lib/ico"
+require_relative "lib/ping"
+require_relative "lib/ping_manager"
 require_relative "lib/broadcast_server"
 require_relative "lib/hardware_survey"
 require_relative "lib/game_settings"
 require_relative "lib/websocket_client"
-require_relative "lib/background_worker"
+require_relative "lib/network_manager"
+require_relative "lib/network_manager/http_client"
 require_relative "lib/application_manager"
 require_relative "lib/application_manager/manifest"
 require_relative "lib/application_manager/status"
@@ -176,27 +179,18 @@ end
 
 logger.info(W3DHub::LOG_TAG) { "W3D Hub Linux Launcher v#{W3DHub::VERSION}" }
 
-Thread.new do
-  W3DHub::BackgroundWorker.create
-end
-
-until W3DHub::BackgroundWorker.alive?
-  sleep 0.1
-end
-
 logger.info(W3DHub::LOG_TAG) { "Launching window..." }
 # W3DHub::Window.new(width: 980, height: 720, borderless: false, resizable: true).show unless defined?(Ocra)
 W3DHub::Window.new(width: 1280, height: 800, borderless: false, resizable: true).show unless defined?(Ocra)
 # W3DHub::Window.new(width: 1920, height: 1080, borderless: false, resizable: true).show unless defined?(Ocra)
-W3DHub::BackgroundWorker.shutdown!
 
-worker_soft_halt = Gosu.milliseconds
+# worker_soft_halt = Gosu.milliseconds
 
-# Wait for BackgroundWorker to return
+# # Wait for BackgroundWorker to return
-while W3DHub::BackgroundWorker.alive?
+# while W3DHub::BackgroundWorker.alive?
-  W3DHub::BackgroundWorker.kill! if Gosu.milliseconds - worker_soft_halt >= 1_000
+#   W3DHub::BackgroundWorker.kill! if Gosu.milliseconds - worker_soft_halt >= 1_000
 
-  sleep 0.1
+#   sleep 0.1
-end
+# end
 
 W3DHub::LOGGER&.close