From 56507d7ac27d0562689ea7c505fa026ecc38494f Mon Sep 17 00:00:00 2001 From: cbdev Date: Sat, 2 May 2020 16:55:13 +0200 Subject: Implement keyboard output for wininput --- backends/wininput.md | 114 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 backends/wininput.md (limited to 'backends/wininput.md') diff --git a/backends/wininput.md b/backends/wininput.md new file mode 100644 index 0000000..672a24e --- /dev/null +++ b/backends/wininput.md @@ -0,0 +1,114 @@ +### The `wininput` backend + +This backend allows using the mouse and keyboard as input and output channels on a Windows system. +For example, it can be used to create hotkey-like behaviour (by reading keyboard input) or to control +a computer remotely. + +As Windows merges all keyboard and mouse input into a single data stream, no fine-grained per-device +access (as is available under Linux) is possible. + +#### Global configuration + +This backend does not take any global configuration. + +#### Instance configuration + +This backend does not take any instance-specific configuration. + +#### Channel specification + +The mouse is exposed as two channels for the position (with the origin being the upper-left corner of the desktop) + +* `mouse.x` +* `mouse.y` + +as well as one channel per mouse button + +* `mouse.lmb` (Left mouse button) +* `mouse.rmb` (Right mouse button) +* `mouse.mmb` (Middle mouse button) +* `mouse.xmb1` (Extra mouse button 1) +* `mouse.xmb2` (Extra mouse button 2) + +All keys that have an [assigned virtual keycode](https://docs.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes) +are mappable as MIDIMonster channels using the syntax `key.`, with *keyname* being one of the following specifiers: + +* One of the keynames listed below (e.g., `key.enter`) +* For "simple" keys (A-z, 0-9, etc), simply the key glyph (e.g. `key.a`) +* A hexadecimal number specifying the virtual keycode + +Keys are pressed once the normalized event value is greater than `0.9`, and released if under that. + +The following keynames are defined in an internal mapping table: + +| Key name | Description | +|-------------------------------|-----------------------| +| `backspace` | | +| `tab` | | +| `clear` | | +| `enter` | | +| `shift` | | +| `control` | | +| `alt` | | +| `capslock` | | +| `escape` | | +| `space`, | | +| `pageup`, `pagedown` | | +| `end` | | +| `home` | | +| `pause` | | +| `numlock` | | +| `scrolllock` | | +| `insert` | | +| `delete` | | +| `printscreen` | | +| `up`, `down`, `left`, `right` | | +| `select` | | +| `print` | | +| `execute` | | +| `help` | | +| `apps` | | +| `sleep` | | +| `num0` - `num9` | | +| `multiply` | | +| `plus` | | +| `comma` | | +| `minus` | | +| `dot` | | +| `divide` | | +| `f1` - `f24` | | +| `lwin`, `rwin` | | +| `lshift`, `rshift` | | +| `lctrl, `rctrl` | | +| `lmenu`, `rmenu` | | +| `previous`, `next` | Browser controls | +| `refresh` | Browser controls | +| `stop` | Browser controls | +| `search` | Browser controls | +| `favorites` | Browser controls | +| `homepage` | Browser controls | +| `mute` | | +| `voldown`, `volup` | | +| `nexttrack`, `prevtrack` | | +| `stopmedia`, `togglemedia` | | +| `mediaselect` | | +| `mail` | | +| `app1`, `app2` | | +| `zoom` | | + +Example mappings: +``` +generator.x > wi1.mouse.x +input.a > wi1.key.a +input.X > wi1.key.escape +``` + +#### Known bugs / problems + +Keyboard and mouse input is subject to UIPI. You can not send input to applications that run at a higher +privilege level than the MIDIMonster. + +Some antivirus applications may detect this backend as problematic file, because it uses the same system +interfaces to read keyboard and mouse input as any malicious application would. While it is definitely +possible to configure the MIDIMonster to do malicious things, the code itself does not log anything. +You can verify this by reading the backend code yourself. -- cgit v1.2.3 From f9886f3b06ce8e32aea893208646053f93a00a6c Mon Sep 17 00:00:00 2001 From: cbdev Date: Sat, 2 May 2020 17:20:13 +0200 Subject: Reformat wininput table --- backends/wininput.md | 94 +++++++++++++++++++--------------------------------- 1 file changed, 34 insertions(+), 60 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/wininput.md b/backends/wininput.md index 672a24e..6455dd1 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -24,11 +24,11 @@ The mouse is exposed as two channels for the position (with the origin being the as well as one channel per mouse button -* `mouse.lmb` (Left mouse button) -* `mouse.rmb` (Right mouse button) -* `mouse.mmb` (Middle mouse button) -* `mouse.xmb1` (Extra mouse button 1) -* `mouse.xmb2` (Extra mouse button 2) +* `mouse.lmb`: Left mouse button +* `mouse.rmb`: Right mouse button +* `mouse.mmb`: Middle mouse button +* `mouse.xmb1`: Extra mouse button 1 +* `mouse.xmb2`: Extra mouse button 2 All keys that have an [assigned virtual keycode](https://docs.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes) are mappable as MIDIMonster channels using the syntax `key.`, with *keyname* being one of the following specifiers: @@ -41,60 +41,34 @@ Keys are pressed once the normalized event value is greater than `0.9`, and rele The following keynames are defined in an internal mapping table: -| Key name | Description | -|-------------------------------|-----------------------| -| `backspace` | | -| `tab` | | -| `clear` | | -| `enter` | | -| `shift` | | -| `control` | | -| `alt` | | -| `capslock` | | -| `escape` | | -| `space`, | | -| `pageup`, `pagedown` | | -| `end` | | -| `home` | | -| `pause` | | -| `numlock` | | -| `scrolllock` | | -| `insert` | | -| `delete` | | -| `printscreen` | | -| `up`, `down`, `left`, `right` | | -| `select` | | -| `print` | | -| `execute` | | -| `help` | | -| `apps` | | -| `sleep` | | -| `num0` - `num9` | | -| `multiply` | | -| `plus` | | -| `comma` | | -| `minus` | | -| `dot` | | -| `divide` | | -| `f1` - `f24` | | -| `lwin`, `rwin` | | -| `lshift`, `rshift` | | -| `lctrl, `rctrl` | | -| `lmenu`, `rmenu` | | -| `previous`, `next` | Browser controls | -| `refresh` | Browser controls | -| `stop` | Browser controls | -| `search` | Browser controls | -| `favorites` | Browser controls | -| `homepage` | Browser controls | -| `mute` | | -| `voldown`, `volup` | | -| `nexttrack`, `prevtrack` | | -| `stopmedia`, `togglemedia` | | -| `mediaselect` | | -| `mail` | | -| `app1`, `app2` | | -| `zoom` | | +| Key name | Description | Key name | Description | +|-------------------------------|-----------------------|-------------------------------|-----------------------| +| `backspace` | | `tab` | | +| `clear` | | `enter` | | +| `shift` | | `control` | | +| `alt` | | `capslock` | | +| `escape` | | `space` | | +| `pageup`, `pagedown` | | `end` | | +| `home` | | `pause` | | +| `numlock` | | `scrolllock` | | +| `insert` | | `delete` | | +| `printscreen` | | `up`, `down`, `left`, `right` | | +| `select` | | `print` | | +| `execute` | | `help` | | +| `apps` | | `sleep` | | +| `num0` - `num9` | | `multiply` | | +| `plus` | | `comma` | | +| `minus` | | `dot` | | +| `divide` | | `f1` - `f24` | | +| `lwin`, `rwin` | | `lshift`, `rshift` | | +| `lctrl, `rctrl` | | `lmenu`, `rmenu` | | +| `previous`, `next` | Browser controls | `refresh` | Browser controls | +| `stop` | Browser controls | `search` | Browser controls | +| `favorites` | Browser controls | `homepage` | Browser controls | +| `mute` | | `voldown`, `volup` | | +| `nexttrack`, `prevtrack` | | `stopmedia`, `togglemedia` | | +| `mediaselect` | | `mail` | | +| `app1`, `app2` | | `zoom` | | Example mappings: ``` @@ -108,7 +82,7 @@ input.X > wi1.key.escape Keyboard and mouse input is subject to UIPI. You can not send input to applications that run at a higher privilege level than the MIDIMonster. -Some antivirus applications may detect this backend as problematic file, because it uses the same system +Some antivirus applications may detect this backend as problematic because it uses the same system interfaces to read keyboard and mouse input as any malicious application would. While it is definitely possible to configure the MIDIMonster to do malicious things, the code itself does not log anything. You can verify this by reading the backend code yourself. -- cgit v1.2.3 From eabdc18aa2209f3526d1068510991ce7f17a4fe6 Mon Sep 17 00:00:00 2001 From: cbdev Date: Mon, 4 May 2020 20:25:00 +0200 Subject: Implement wininput input direction --- backends/wininput.c | 223 ++++++++++++++++++++++++++++++++++++++++++++++----- backends/wininput.h | 13 +-- backends/wininput.md | 6 +- 3 files changed, 215 insertions(+), 27 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/wininput.c b/backends/wininput.c index 46151d4..0c7e490 100644 --- a/backends/wininput.c +++ b/backends/wininput.c @@ -4,6 +4,9 @@ #include #include "wininput.h" +//TODO check whether feedback elimination is required +//TODO refactor & simplify + static key_info keys[] = { {VK_LBUTTON, "lmb", button}, {VK_RBUTTON, "rmb", button}, {VK_MBUTTON, "mmb", button}, {VK_XBUTTON1, "xmb1", button}, {VK_XBUTTON2, "xmb2", button}, @@ -53,11 +56,23 @@ static key_info keys[] = { {VK_ZOOM, "zoom"} }; +static struct { + int virtual_x, virtual_y, virtual_width, virtual_height; + uint16_t mouse_x, mouse_y; + size_t requests; + wininput_request* request; + uint32_t interval; +} cfg = { + .requests = 0, + .interval = 50 +}; + MM_PLUGIN_API int init(){ backend wininput = { .name = BACKEND_NAME, .conf = wininput_configure, .create = wininput_instance, + .interval = wininput_interval, .conf_instance = wininput_configure_instance, .channel = wininput_channel, .handle = wininput_set, @@ -79,8 +94,17 @@ MM_PLUGIN_API int init(){ return 0; } +static uint32_t wininput_interval(){ + return cfg.interval; +} + static int wininput_configure(char* option, char* value){ - LOG("The backend does not take any global configuration"); + if(!strcmp(option, "interval")){ + cfg.interval = strtoul(value, NULL, 0); + return 0; + } + + LOGPF("Unknown backend configuration option %s", option); return 1; } @@ -90,18 +114,57 @@ static int wininput_configure_instance(instance* inst, char* option, char* value } static int wininput_instance(instance* inst){ - wininput_instance_data* data = calloc(1, sizeof(wininput_instance_data)); - if(!data){ + return 0; +} + +static int wininput_subscribe(wininput_channel_ident ident, channel* chan){ + size_t u, n; + + //find an existing request + for(u = 0; u < cfg.requests; u++){ + if(cfg.request[u].ident.label == ident.label){ + break; + } + } + + if(u == cfg.requests){ + //create a new request + cfg.request = realloc(cfg.request, (cfg.requests + 1) * sizeof(wininput_request)); + if(!cfg.request){ + cfg.requests = 0; + LOG("Failed to allocate memory"); + return 1; + } + + cfg.request[u].ident.label = ident.label; + cfg.request[u].channels = 0; + cfg.request[u].channel = NULL; + cfg.request[u].state = 0; + cfg.requests++; + } + + //check if already in subscriber list + for(n = 0; n < cfg.request[u].channels; n++){ + if(cfg.request[u].channel[n] == chan){ + return 0; + } + } + + //add to subscriber list + cfg.request[u].channel = realloc(cfg.request[u].channel, (cfg.request[u].channels + 1) * sizeof(channel*)); + if(!cfg.request[u].channel){ + cfg.request[u].channels = 0; LOG("Failed to allocate memory"); return 1; } - - inst->impl = data; + cfg.request[u].channel[n] = chan; + cfg.request[u].channels++; return 0; } static channel* wininput_channel(instance* inst, char* spec, uint8_t flags){ size_t u; + channel* chan = NULL; uint16_t scancode = 0; char* token = spec; wininput_channel_ident ident = { @@ -129,6 +192,11 @@ static channel* wininput_channel(instance* inst, char* spec, uint8_t flags){ break; } } + + if(u == sizeof(keys) / sizeof(keys[0])){ + LOGPF("Unknown mouse control %s", token); + return NULL; + } } } else if(!strncmp(spec, "key.", 4)){ @@ -181,12 +249,26 @@ static channel* wininput_channel(instance* inst, char* spec, uint8_t flags){ } if(ident.label){ - return mm_channel(inst, ident.label, 1); + chan = mm_channel(inst, ident.label, 1); + if(chan && (flags & mmchannel_input) && wininput_subscribe(ident, chan)){ + return NULL; + } + return chan; } return NULL; } -static INPUT wininput_event_mouse(wininput_instance_data* data, uint8_t channel, uint8_t control, double value){ +//for some reason, sendinput only takes "normalized absolute coordinates", which are never again used in the API +static void wininput_mouse_normalize(long* x, long* y){ + //TODO this needs to take a possible origin offset into account + long normalized_x = (double) (*x) * (65535.0f / (double) cfg.virtual_width); + long normalized_y = (double) (*y) * (65535.0f / (double) cfg.virtual_height); + + *x = normalized_x; + *y = normalized_y; +} + +static INPUT wininput_event_mouse(uint8_t channel, uint8_t control, double value){ DWORD flags_down = 0, flags_up = 0; INPUT ev = { .type = INPUT_MOUSE @@ -194,15 +276,15 @@ static INPUT wininput_event_mouse(wininput_instance_data* data, uint8_t channel, if(channel == position){ if(control){ - data->mouse.y = value * 0xFFFF; + cfg.mouse_y = value * 0xFFFF; } else{ - data->mouse.x = value * 0xFFFF; + cfg.mouse_x = value * 0xFFFF; } - ev.mi.dwFlags |= MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE; - ev.mi.dx = data->mouse.x; - ev.mi.dy = data->mouse.y; + ev.mi.dwFlags |= MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE | MOUSEEVENTF_VIRTUALDESK; + ev.mi.dx = cfg.mouse_x; + ev.mi.dy = cfg.mouse_y; } if(channel == button){ switch(control){ @@ -237,7 +319,7 @@ static INPUT wininput_event_mouse(wininput_instance_data* data, uint8_t channel, return ev; } -static INPUT wininput_event_keyboard(wininput_instance_data* data, uint8_t channel, uint8_t control, double value){ +static INPUT wininput_event_keyboard(uint8_t channel, uint8_t control, double value){ INPUT ev = { .type = INPUT_KEYBOARD }; @@ -256,7 +338,6 @@ static int wininput_set(instance* inst, size_t num, channel** c, channel_value* wininput_channel_ident ident = { .label = 0 }; - wininput_instance_data* data = (wininput_instance_data*) inst->impl; size_t n = 0, offset = 0; INPUT events[500]; @@ -268,10 +349,10 @@ static int wininput_set(instance* inst, size_t num, channel** c, channel_value* for(n = 0; n + offset < num; n++){ ident.label = c[n + offset]->ident; if(ident.fields.type == mouse){ - events[n] = wininput_event_mouse(data, ident.fields.channel, ident.fields.control, v[n + offset].normalised); + events[n] = wininput_event_mouse(ident.fields.channel, ident.fields.control, v[n + offset].normalised); } else if(ident.fields.type == keyboard){ - events[n] = wininput_event_keyboard(data, ident.fields.channel, ident.fields.control, v[n + offset].normalised); + events[n] = wininput_event_keyboard(ident.fields.channel, ident.fields.control, v[n + offset].normalised); } else{ n--; @@ -289,21 +370,123 @@ static int wininput_set(instance* inst, size_t num, channel** c, channel_value* } static int wininput_handle(size_t num, managed_fd* fds){ - //TODO + channel_value val = { + .normalised = 0 + }; + uint8_t mouse_updated = 0, synthesize_off = 0, push_event = 0; + uint16_t key_state = 0; + POINT cursor_position; + size_t u = 0, n; + + for(u = 0; u < cfg.requests; u++){ + synthesize_off = 0; + push_event = 0; + val.normalised = 0; + + if(cfg.request[u].ident.fields.type == mouse + && cfg.request[u].ident.fields.channel == position){ + if(!mouse_updated){ + //update mouse coordinates + if(!GetCursorPos(&cursor_position)){ + LOG("Failed to update mouse position"); + continue; + } + wininput_mouse_normalize(&cursor_position.x, &cursor_position.y); + mouse_updated = 1; + if(cfg.mouse_x != cursor_position.x + || cfg.mouse_y != cursor_position.y){ + cfg.mouse_x = cursor_position.x; + cfg.mouse_y = cursor_position.y; + mouse_updated = 2; + } + } + + val.normalised = (double) cfg.mouse_x / (double) 0xFFFF; + if(cfg.request[u].ident.fields.control){ + val.normalised = (double) cfg.mouse_y / (double) 0xFFFF; + } + + if(mouse_updated == 2){ + push_event = 1; + } + } + else{ + //check key state + key_state = GetAsyncKeyState(cfg.request[u].ident.fields.control); + if(key_state == 1){ + //pressed and released? + synthesize_off = 1; + } + if((key_state & ~1) != cfg.request[u].state){ + //key state changed + if(key_state){ + val.normalised = 1.0; + } + cfg.request[u].state = key_state & ~1; + push_event = 1; + } + } + + if(push_event){ + //push current value to all channels + DBGPF("Pushing event %f on request %" PRIsize_t, val.normalised, u); + for(n = 0; n < cfg.request[u].channels; n++){ + mm_channel_event(cfg.request[u].channel[n], val); + } + + if(synthesize_off){ + val.normalised = 0; + //push synthesized value to all channels + DBGPF("Synthesizing event %f on request %" PRIsize_t, val.normalised, u); + for(n = 0; n < cfg.request[u].channels; n++){ + mm_channel_event(cfg.request[u].channel[n], val); + } + } + } + } return 0; } static int wininput_start(size_t n, instance** inst){ - //TODO + POINT cursor_position; + + //if no input requested, don't request polling + if(!cfg.requests){ + cfg.interval = 0; + } + + //read virtual desktop extents for later normalization + cfg.virtual_width = GetSystemMetrics(SM_CXVIRTUALSCREEN); + cfg.virtual_height = GetSystemMetrics(SM_CYVIRTUALSCREEN); + cfg.virtual_x = GetSystemMetrics(SM_XVIRTUALSCREEN); + cfg.virtual_y = GetSystemMetrics(SM_YVIRTUALSCREEN); + DBGPF("Virtual screen is %dx%d with offset %dx%d", cfg.virtual_width, cfg.virtual_height, cfg.virtual_x, cfg.virtual_y); + + //initialize mouse position + if(!GetCursorPos(&cursor_position)){ + LOG("Failed to read initial mouse position"); + return 1; + } + + DBGPF("Current mouse coordinates: %dx%d (%04Xx%04X)", cursor_position.x, cursor_position.y, cursor_position.x, cursor_position.y); + wininput_mouse_normalize(&cursor_position.x, &cursor_position.y); + DBGPF("Current normalized mouse position: %04Xx%04X", cursor_position.x, cursor_position.y); + cfg.mouse_x = cursor_position.x; + cfg.mouse_y = cursor_position.y; + + DBGPF("Tracking %" PRIsize_t " input requests", cfg.requests); return 0; } static int wininput_shutdown(size_t n, instance** inst){ size_t u; - for(u = 0; u < n; u++){ - free(inst[u]->impl); + for(u = 0; u < cfg.requests; u++){ + free(cfg.request[u].channel); } + free(cfg.request); + cfg.request = NULL; + cfg.requests = 0; LOG("Backend shut down"); return 0; diff --git a/backends/wininput.h b/backends/wininput.h index ef817e8..0318724 100644 --- a/backends/wininput.h +++ b/backends/wininput.h @@ -5,6 +5,7 @@ static int wininput_configure(char* option, char* value); static int wininput_configure_instance(instance* inst, char* option, char* value); static int wininput_instance(instance* inst); static channel* wininput_channel(instance* inst, char* spec, uint8_t flags); +static uint32_t wininput_interval(); static int wininput_set(instance* inst, size_t num, channel** c, channel_value* v); static int wininput_handle(size_t num, managed_fd* fds); static int wininput_start(size_t n, instance** inst); @@ -42,9 +43,9 @@ typedef union { uint64_t label; } wininput_channel_ident; -typedef struct { - struct { - uint16_t x; - uint16_t y; - } mouse; -} wininput_instance_data; +typedef struct /*_input_request*/ { + wininput_channel_ident ident; + size_t channels; + channel** channel; + uint16_t state; +} wininput_request; diff --git a/backends/wininput.md b/backends/wininput.md index 6455dd1..6ead158 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -61,7 +61,7 @@ The following keynames are defined in an internal mapping table: | `minus` | | `dot` | | | `divide` | | `f1` - `f24` | | | `lwin`, `rwin` | | `lshift`, `rshift` | | -| `lctrl, `rctrl` | | `lmenu`, `rmenu` | | +| `lctrl`, `rctrl` | | `lmenu`, `rmenu` | | | `previous`, `next` | Browser controls | `refresh` | Browser controls | | `stop` | Browser controls | `search` | Browser controls | | `favorites` | Browser controls | `homepage` | Browser controls | @@ -82,6 +82,10 @@ input.X > wi1.key.escape Keyboard and mouse input is subject to UIPI. You can not send input to applications that run at a higher privilege level than the MIDIMonster. +Due to inconsistencies in the Windows API, mouse position input and output may differ for the same cursor location. +This may be correlated with the use and arrangement of multi-monitor desktops. If you encounter problems with either +receiving or sending mouse positions, please include a description of your monitor alignment in the issue. + Some antivirus applications may detect this backend as problematic because it uses the same system interfaces to read keyboard and mouse input as any malicious application would. While it is definitely possible to configure the MIDIMonster to do malicious things, the code itself does not log anything. -- cgit v1.2.3 From 5e4c3fe5a81f548243aab01a42973333b559004f Mon Sep 17 00:00:00 2001 From: cbdev Date: Thu, 11 Jun 2020 12:27:57 +0200 Subject: Query joystick buttons --- backends/maweb.c | 2 +- backends/wininput.c | 87 ++++++++++++++++++++++++++++++++++++++++++++-------- backends/wininput.h | 5 ++- backends/wininput.md | 5 ++- 4 files changed, 84 insertions(+), 15 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/maweb.c b/backends/maweb.c index 7829ec4..33d2b7e 100644 --- a/backends/maweb.c +++ b/backends/maweb.c @@ -669,7 +669,7 @@ static void maweb_disconnect(instance* inst){ char xmit_buffer[MAWEB_XMIT_CHUNK]; if(data->fd){ - //close the session if one is active + //close the session if one is active if(data->session > 0){ snprintf(xmit_buffer, sizeof(xmit_buffer), "{\"requestType\":\"close\",\"session\":%" PRIu64 "}", data->session); maweb_send_frame(inst, ws_text, (uint8_t*) xmit_buffer, strlen(xmit_buffer)); diff --git a/backends/wininput.c b/backends/wininput.c index 71e9855..627891e 100644 --- a/backends/wininput.c +++ b/backends/wininput.c @@ -1,5 +1,5 @@ #define BACKEND_NAME "wininput" -#define DEBUG +//#define DEBUG #include #include "wininput.h" @@ -7,6 +7,7 @@ #include //TODO check whether feedback elimination is required +//TODO might want to store virtual desktop extents in request->limit static key_info keys[] = { {VK_LBUTTON, "lmb", button}, {VK_RBUTTON, "rmb", button}, {VK_MBUTTON, "mmb", button}, @@ -57,6 +58,17 @@ static key_info keys[] = { {VK_ZOOM, "zoom"} }; +//this monstrosity is necessary because not only are the buttons not a simple bitmask, the bits are also partially reused. +//i get why they replaced this heap of trash API, but the replacement would require me to jump through even more hoops. +static uint32_t button_masks[32] = {JOY_BUTTON1, JOY_BUTTON2, JOY_BUTTON3, JOY_BUTTON4, + JOY_BUTTON5, JOY_BUTTON6, JOY_BUTTON7, JOY_BUTTON8, + JOY_BUTTON9, JOY_BUTTON10, JOY_BUTTON11, JOY_BUTTON12, + JOY_BUTTON13, JOY_BUTTON14, JOY_BUTTON15, JOY_BUTTON16, + JOY_BUTTON17, JOY_BUTTON18, JOY_BUTTON19, JOY_BUTTON20, + JOY_BUTTON21, JOY_BUTTON22, JOY_BUTTON23, JOY_BUTTON24, + JOY_BUTTON25, JOY_BUTTON26, JOY_BUTTON27, JOY_BUTTON28, + JOY_BUTTON29, JOY_BUTTON30, JOY_BUTTON31, JOY_BUTTON32}; + static struct { int virtual_x, virtual_y, virtual_width, virtual_height; long mouse_x, mouse_y; @@ -159,7 +171,7 @@ static int wininput_subscribe(uint64_t ident, channel* chan){ cfg.request[u].ident.label = ident; cfg.request[u].channels = 0; cfg.request[u].channel = NULL; - cfg.request[u].state = 0; + cfg.request[u].state = cfg.request[u].min = cfg.request[u].max = 0; cfg.requests++; } @@ -276,7 +288,7 @@ static uint64_t wininput_channel_joystick(instance* inst, char* spec, uint8_t fl if(strlen(token) == 1 || !strcmp(token, "pov")){ if(strchr(axes, token[0])){ ident.fields.channel = position; - ident.fields.control = (controller << 8) | token[0]; + ident.fields.control = ((controller - 1) << 8) | token[0]; return ident.label; } @@ -511,13 +523,25 @@ static int wininput_handle(size_t num, managed_fd* fds){ if(cfg.request[u].ident.fields.channel == button){ //button query if(joy_info.dwFlags & JOY_RETURNBUTTONS){ - //TODO handle button requests + key_state = (joy_info.dwButtons & button_masks[(cfg.request[u].ident.fields.control & 0xFF)]) > 0 ? 1 : 0; + if(key_state != cfg.request[u].state){ + if(key_state){ + val.normalised = 1.0; + } + cfg.request[u].state = key_state; + push_event = 1; + DBGPF("Joystick %d button %d: %d", + cfg.request[u].ident.fields.control >> 8, + cfg.request[u].ident.fields.control & 0xFF, + key_state); + } } else{ LOGPF("No button data received for joystick %d", cfg.request[u].ident.fields.control >> 8); } } else{ + //TODO handle axis requests } } @@ -542,17 +566,11 @@ static int wininput_handle(size_t num, managed_fd* fds){ return 0; } -static int wininput_start(size_t n, instance** inst){ - size_t u; - POINT cursor_position; +static void wininput_start_joystick(){ + size_t u, p; JOYINFOEX joy_info; JOYCAPS joy_caps; - //if no input requested, don't request polling - if(!cfg.requests){ - cfg.interval = 0; - } - DBGPF("This system supports a maximum of %u joysticks", joyGetNumDevs()); for(u = 0; u < joyGetNumDevs(); u++){ joy_info.dwSize = sizeof(joy_info); @@ -560,12 +578,57 @@ static int wininput_start(size_t n, instance** inst){ if(joyGetPosEx(u, &joy_info) == JOYERR_NOERROR){ if(joyGetDevCaps(u, &joy_caps, sizeof(joy_caps)) == JOYERR_NOERROR){ LOGPF("Joystick %" PRIsize_t " (%s) is available for input", u + 1, joy_caps.szPname ? joy_caps.szPname : "unknown model"); + for(p = 0; p < cfg.requests; p++){ + if(cfg.request[p].ident.fields.type == joystick + && cfg.request[p].ident.fields.channel == position + && (cfg.request[p].ident.fields.control >> 8) == u){ + //this looks really dumb, but the structure is defined in a way that prevents us from doing anything clever here + switch(cfg.request[p].ident.fields.control & 0xFF){ + case 'x': + cfg.request[p].min = joy_caps.wXmin; + cfg.request[p].max = joy_caps.wXmax; + break; + case 'y': + cfg.request[p].min = joy_caps.wYmin; + cfg.request[p].max = joy_caps.wYmax; + break; + case 'z': + cfg.request[p].min = joy_caps.wZmin; + cfg.request[p].max = joy_caps.wZmax; + break; + case 'r': + cfg.request[p].min = joy_caps.wRmin; + cfg.request[p].max = joy_caps.wRmax; + break; + case 'u': + cfg.request[p].min = joy_caps.wUmin; + cfg.request[p].max = joy_caps.wUmax; + break; + case 'v': + cfg.request[p].min = joy_caps.wVmin; + cfg.request[p].max = joy_caps.wVmax; + break; + } + DBGPF("Updated limits on request %" PRIsize_t " to %" PRIu32 " / %" PRIu32, p, cfg.request[p].min, cfg.request[p].max); + } + } } else{ LOGPF("Joystick %" PRIsize_t " available for input, but no capabilities reported", u + 1); } } } +} + +static int wininput_start(size_t n, instance** inst){ + POINT cursor_position; + + //if no input requested, don't request polling + if(!cfg.requests){ + cfg.interval = 0; + } + + wininput_start_joystick(); //read virtual desktop extents for later normalization cfg.virtual_width = GetSystemMetrics(SM_CXVIRTUALSCREEN); diff --git a/backends/wininput.h b/backends/wininput.h index 0318724..6ec9f46 100644 --- a/backends/wininput.h +++ b/backends/wininput.h @@ -47,5 +47,8 @@ typedef struct /*_input_request*/ { wininput_channel_ident ident; size_t channels; channel** channel; - uint16_t state; + uint32_t state; + + //used for jostick axes + uint32_t min, max; } wininput_request; diff --git a/backends/wininput.md b/backends/wininput.md index 6ead158..6e665bb 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -79,8 +79,11 @@ input.X > wi1.key.escape #### Known bugs / problems +Joysticks can only be used as input to the MIDIMonster, as Windows does not provide a method to emulate +Joystick input from user space. This is unlikely to change. + Keyboard and mouse input is subject to UIPI. You can not send input to applications that run at a higher -privilege level than the MIDIMonster. +privilege level than the MIDIMonster. This limitation is by design and will not change. Due to inconsistencies in the Windows API, mouse position input and output may differ for the same cursor location. This may be correlated with the use and arrangement of multi-monitor desktops. If you encounter problems with either -- cgit v1.2.3 From 7b3245de94d5e9453ec1f817aa810f6bd3e64c28 Mon Sep 17 00:00:00 2001 From: cbdev Date: Sat, 13 Jun 2020 10:32:43 +0200 Subject: Implement joystick axis input --- backends/wininput.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++---- backends/wininput.md | 27 +++++++++++++++++++++++++ 2 files changed, 80 insertions(+), 4 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/wininput.c b/backends/wininput.c index 627891e..876e276 100644 --- a/backends/wininput.c +++ b/backends/wininput.c @@ -510,7 +510,7 @@ static int wininput_handle(size_t num, managed_fd* fds){ else if(cfg.request[u].ident.fields.type == joystick){ if(cfg.request[u].ident.fields.control >> 8 != current_joystick){ joy_info.dwSize = sizeof(joy_info); - joy_info.dwFlags = JOY_RETURNALL; + joy_info.dwFlags = JOY_RETURNALL | JOY_RETURNPOVCTS; if(joyGetPosEx((cfg.request[u].ident.fields.control >> 8) - 1, &joy_info) != JOYERR_NOERROR){ LOGPF("Failed to query joystick %d", cfg.request[u].ident.fields.control >> 8); //early exit because other joystick probably won't be connected either (though this may be wrong) @@ -523,7 +523,7 @@ static int wininput_handle(size_t num, managed_fd* fds){ if(cfg.request[u].ident.fields.channel == button){ //button query if(joy_info.dwFlags & JOY_RETURNBUTTONS){ - key_state = (joy_info.dwButtons & button_masks[(cfg.request[u].ident.fields.control & 0xFF)]) > 0 ? 1 : 0; + key_state = (joy_info.dwButtons & button_masks[(cfg.request[u].ident.fields.control & 0xFF) - 1]) > 0 ? 1 : 0; if(key_state != cfg.request[u].state){ if(key_state){ val.normalised = 1.0; @@ -541,12 +541,61 @@ static int wininput_handle(size_t num, managed_fd* fds){ } } else{ + if(!cfg.request[u].max){ + cfg.request[u].max = 0xFFFF; + } + val.raw.u64 = cfg.request[u].state; - //TODO handle axis requests + //axis requests, every single access to these structures is stupid. + switch(cfg.request[u].ident.fields.control & 0xFF){ + case 'x': + if(joy_info.dwFlags & JOY_RETURNX){ + val.raw.u64 = joy_info.dwXpos; + } + break; + case 'y': + if(joy_info.dwFlags & JOY_RETURNY){ + val.raw.u64 = joy_info.dwYpos; + } + break; + case 'z': + if(joy_info.dwFlags & JOY_RETURNZ){ + val.raw.u64 = joy_info.dwZpos; + } + break; + case 'r': + if(joy_info.dwFlags & JOY_RETURNR){ + val.raw.u64 = joy_info.dwRpos; + } + break; + case 'u': + if(joy_info.dwFlags & JOY_RETURNU){ + val.raw.u64 = joy_info.dwUpos; + } + break; + case 'v': + if(joy_info.dwFlags & JOY_RETURNV){ + val.raw.u64 = joy_info.dwVpos; + } + break; + case 'p': + if(joy_info.dwFlags & (JOY_RETURNPOV | JOY_RETURNPOVCTS)){ + val.raw.u64 = joy_info.dwPOV; + } + break; + } + + if(val.raw.u64 != cfg.request[u].state){ + val.normalised = (double) (val.raw.u64 - cfg.request[u].min) / (double) (cfg.request[u].max - cfg.request[u].min); + cfg.request[u].state = val.raw.u64; + push_event = 1; + } } } if(push_event){ + //clamp value just to be safe + val.normalised = clamp(val.normalised, 1.0, 0.0); //push current value to all channels DBGPF("Pushing event %f on request %" PRIsize_t, val.normalised, u); for(n = 0; n < cfg.request[u].channels; n++){ @@ -609,7 +658,7 @@ static void wininput_start_joystick(){ cfg.request[p].max = joy_caps.wVmax; break; } - DBGPF("Updated limits on request %" PRIsize_t " to %" PRIu32 " / %" PRIu32, p, cfg.request[p].min, cfg.request[p].max); + DBGPF("Updated limits on request %" PRIsize_t " (%c) to %" PRIu32 " / %" PRIu32, p, cfg.request[p].ident.fields.control & 0xFF, cfg.request[p].min, cfg.request[p].max); } } } diff --git a/backends/wininput.md b/backends/wininput.md index 6e665bb..bcf6a1b 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -77,6 +77,33 @@ input.a > wi1.key.a input.X > wi1.key.escape ``` +Joystick and gamepad controllers with up to 32 buttons and 6 axes plus POV hat can be mapped as inputs to the +MIDIMonster. When starting up, the MIDIMonster will output a list of all connected and usable game controllers. + +Controllers can be mapped using the syntax + +* `joy.` for axes, where `` is the ID of the controller and `` is one of + * `x`, `y`: Main joystick / analog controller axes + * `z`: Third axis / joystick rotation + * `r`: Fourth axis / Rudder controller / Slider + * `u`, `v`: non-specific fifth/sixth axis +* `joy.button` for buttons, with `` again being the controller ID and `b` being the button number between + 1 and 32 (the maximum supported by Windows) + +Use the Windows game controller input calibration and configuration tool to identify the axes and button IDs +relevant to your controller. + +For button channels, the channel value will either be `0` or `1.0`, for axis channels it will be the normalized +value of the axis (with calibration offsets applied), with the exception of the POV axis, where the channel value +will be in some way correlated with the direction of view. + +Example mappings: +``` +input.joy1.x > movinghead.pan +input.joy1.y > movinghead.tilt +input.joy1.button1 > movinghead.dim +``` + #### Known bugs / problems Joysticks can only be used as input to the MIDIMonster, as Windows does not provide a method to emulate -- cgit v1.2.3 From e131992bbe0893a3e5b79cf9423830ea90b4a4d7 Mon Sep 17 00:00:00 2001 From: cbdev Date: Tue, 18 Aug 2020 06:37:01 +0200 Subject: Implement wininput mouse wheel control (Fixes #65) --- backends/wininput.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++-- backends/wininput.h | 2 +- backends/wininput.md | 15 ++++++++++++++ config.c | 2 +- 4 files changed, 71 insertions(+), 4 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/wininput.c b/backends/wininput.c index 8926782..6d21a76 100644 --- a/backends/wininput.c +++ b/backends/wininput.c @@ -65,9 +65,13 @@ static struct { //sorted in _start wininput_request* request; uint32_t interval; + uint64_t wheel, wheel_max, wheel_delta; + uint8_t wheel_inverted; } cfg = { .requests = 0, - .interval = 50 + .interval = 50, + .wheel_max = 0xFFFF, + .wheel_delta = 1 }; MM_PLUGIN_API int init(){ @@ -120,10 +124,38 @@ static uint32_t wininput_interval(){ } static int wininput_configure(char* option, char* value){ + int64_t parameter = 0; + char* next_token = NULL; + if(!strcmp(option, "interval")){ cfg.interval = strtoul(value, NULL, 0); return 0; } + else if(!strcmp(option, "wheel")){ + parameter = strtoll(value, &next_token, 0); + + cfg.wheel_max = parameter; + if(parameter < 0){ + LOG("Inverting mouse wheel data"); + cfg.wheel_max = -parameter; + cfg.wheel_inverted = 1; + } + else if(!parameter){ + LOGPF("Invalid mouse wheel configuration %s", value); + return 1; + } + + if(next_token && *next_token){ + cfg.wheel = strtoul(next_token, NULL, 0); + } + + return 0; + } + else if(!strcmp(option, "wheeldelta")){ + cfg.wheel_delta = strtoul(value, NULL, 0); + return 0; + } + LOGPF("Unknown backend configuration option %s", option); return 1; @@ -196,6 +228,13 @@ static uint64_t wininput_channel_mouse(instance* inst, char* spec, uint8_t flags ident.fields.channel = position; ident.fields.control = 1; } + else if(!strcmp(spec, "wheel")){ + ident.fields.channel = wheel; + if(flags & mmchannel_input){ + LOG("The mouse wheel can only be used as an output channel"); + return 0; + } + } else{ //check the buttons for(u = 0; u < sizeof(keys) / sizeof(keys[0]); u++){ @@ -354,7 +393,7 @@ static INPUT wininput_event_mouse(uint8_t channel, uint8_t control, double value ev.mi.dx = cfg.mouse_x; ev.mi.dy = cfg.mouse_y; } - if(channel == button){ + else if(channel == button){ switch(control){ case VK_LBUTTON: flags_up |= MOUSEEVENTF_LEFTUP; @@ -383,6 +422,15 @@ static INPUT wininput_event_mouse(uint8_t channel, uint8_t control, double value ev.mi.dwFlags |= flags_up; } } + else if(channel == wheel){ + ev.mi.dwFlags |= MOUSEEVENTF_WHEEL; + ev.mi.mouseData = ((value * cfg.wheel_max) - cfg.wheel) * cfg.wheel_delta; + if(cfg.wheel_inverted){ + ev.mi.mouseData *= -1; + } + DBGPF("Moving wheel %d (invert %d) with delta %d: %d", (value * cfg.wheel_max) - cfg.wheel, cfg.wheel_inverted, cfg.wheel_delta, ev.mi.mouseData); + cfg.wheel = (value * cfg.wheel_max); + } return ev; } @@ -479,6 +527,10 @@ static int wininput_handle(size_t num, managed_fd* fds){ push_event = 1; } } + else if(cfg.request[u].ident.fields.type == mouse + && cfg.request[u].ident.fields.channel == wheel){ + //ignore wheel requests, can't read that + } else if(cfg.request[u].ident.fields.type == keyboard || cfg.request[u].ident.fields.type == mouse){ //check key state diff --git a/backends/wininput.h b/backends/wininput.h index 6ec9f46..0939cc3 100644 --- a/backends/wininput.h +++ b/backends/wininput.h @@ -22,7 +22,7 @@ enum /*wininput_control_channel*/ { keypress = 0, button, position, - //wheel, /*relative*/ + wheel, key_unicode }; diff --git a/backends/wininput.md b/backends/wininput.md index bcf6a1b..87e9321 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -11,6 +11,12 @@ access (as is available under Linux) is possible. This backend does not take any global configuration. +| Option | Example value | Default value | Description | +|---------------|-----------------------|-----------------------|---------------------------------------| +| `interval` | `100` | `50` | Data polling interval in milliseconds. Lower intervals lead to higher CPU load. This value should normally not be changed. | +| `wheel` | `4000 2000` | `65535 0` | Mouse wheel range and optional initial value. To invert the mouse wheel control, specify the range as a negative integer. As the mouse wheel is a relative control, we need to specify a range incoming absolute values are mapped to. This can be used control the wheel resolution and travel size. | +| `wheeldelta` | `20` | `1` | Multiplier for wheel travel | + #### Instance configuration This backend does not take any instance-specific configuration. @@ -30,6 +36,11 @@ as well as one channel per mouse button * `mouse.xmb1`: Extra mouse button 1 * `mouse.xmb2`: Extra mouse button 2 +The (vertical) mouse wheel can be controlled from the MIDIMonster using the `mouse.wheel` channel, but it can not be used +as an input channel due to limitations in the Windows API. All instances share one wheel control (see the section on known +bugs below). The mouse wheel sensitivity can be controlled by adjusting the absolute travel range, its initial value and +a wheel delta multiplier. + All keys that have an [assigned virtual keycode](https://docs.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes) are mappable as MIDIMonster channels using the syntax `key.`, with *keyname* being one of the following specifiers: @@ -120,3 +131,7 @@ Some antivirus applications may detect this backend as problematic because it us interfaces to read keyboard and mouse input as any malicious application would. While it is definitely possible to configure the MIDIMonster to do malicious things, the code itself does not log anything. You can verify this by reading the backend code yourself. + +Since the Windows input system merges all keyboard/mouse input data into one data stream, using multiple +instances of this backend is not necessary or useful. It is still supported for technical reasons. +There may be unexpected side effects when mapping the mouse wheel in multiple instances. diff --git a/config.c b/config.c index 9945319..c1c3124 100644 --- a/config.c +++ b/config.c @@ -545,7 +545,7 @@ static int config_line(char* line){ //find separator separator = strchr(line, '='); if(!separator){ - fprintf(stderr, "Not an assignment: %s\n", line); + fprintf(stderr, "Not an assignment (currently expecting %s configuration): %s\n", line, (parser_state == backend_cfg) ? "backend" : "instance"); return 1; } -- cgit v1.2.3 From 436bcf59c7dcf4acf42f5f812d0e6c4839326757 Mon Sep 17 00:00:00 2001 From: cbdev Date: Tue, 18 Aug 2020 06:40:01 +0200 Subject: Update wininput documentation --- backends/wininput.md | 2 -- 1 file changed, 2 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/wininput.md b/backends/wininput.md index 87e9321..bffa000 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -9,8 +9,6 @@ access (as is available under Linux) is possible. #### Global configuration -This backend does not take any global configuration. - | Option | Example value | Default value | Description | |---------------|-----------------------|-----------------------|---------------------------------------| | `interval` | `100` | `50` | Data polling interval in milliseconds. Lower intervals lead to higher CPU load. This value should normally not be changed. | -- cgit v1.2.3 From dc7303cc07565a725eec19aeff46f64ad4275c21 Mon Sep 17 00:00:00 2001 From: cbdev Date: Wed, 19 Aug 2020 12:46:31 +0200 Subject: wininput sanity check and minor documentation update --- backends/wininput.c | 5 +++++ backends/wininput.md | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) (limited to 'backends/wininput.md') diff --git a/backends/wininput.c b/backends/wininput.c index 6d21a76..1d1c85b 100644 --- a/backends/wininput.c +++ b/backends/wininput.c @@ -149,6 +149,11 @@ static int wininput_configure(char* option, char* value){ cfg.wheel = strtoul(next_token, NULL, 0); } + if(cfg.wheel > cfg.wheel_max){ + LOG("Mouse wheel initial value out of range"); + return 1; + } + return 0; } else if(!strcmp(option, "wheeldelta")){ diff --git a/backends/wininput.md b/backends/wininput.md index bffa000..797d879 100644 --- a/backends/wininput.md +++ b/backends/wininput.md @@ -12,7 +12,7 @@ access (as is available under Linux) is possible. | Option | Example value | Default value | Description | |---------------|-----------------------|-----------------------|---------------------------------------| | `interval` | `100` | `50` | Data polling interval in milliseconds. Lower intervals lead to higher CPU load. This value should normally not be changed. | -| `wheel` | `4000 2000` | `65535 0` | Mouse wheel range and optional initial value. To invert the mouse wheel control, specify the range as a negative integer. As the mouse wheel is a relative control, we need to specify a range incoming absolute values are mapped to. This can be used control the wheel resolution and travel size. | +| `wheel` | `-4000 2000` | `65535 0` | Mouse wheel range and optional initial value. To invert the mouse wheel control, specify the range as a negative integer. As the mouse wheel is a relative control, we need to specify a range incoming absolute values are mapped to. This can be used control the wheel resolution and travel size. | | `wheeldelta` | `20` | `1` | Multiplier for wheel travel | #### Instance configuration @@ -35,11 +35,11 @@ as well as one channel per mouse button * `mouse.xmb2`: Extra mouse button 2 The (vertical) mouse wheel can be controlled from the MIDIMonster using the `mouse.wheel` channel, but it can not be used -as an input channel due to limitations in the Windows API. All instances share one wheel control (see the section on known +as an input channel due to limitations in the Windows API. All instances share one `wheel` control (see the section on known bugs below). The mouse wheel sensitivity can be controlled by adjusting the absolute travel range, its initial value and a wheel delta multiplier. -All keys that have an [assigned virtual keycode](https://docs.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes) +All keyboard keys that have an [assigned virtual keycode](https://docs.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes) are mappable as MIDIMonster channels using the syntax `key.`, with *keyname* being one of the following specifiers: * One of the keynames listed below (e.g., `key.enter`) -- cgit v1.2.3