version 0.10.0

kitty now supports the SGR DIM escape code, which makes text fade into
the background. It works by alpha blending the text color into the
background color. Fixes #446

.ignore

@@ -0,0 +1 @@
+kittens/unicode_input/names.h

.travis.yml

@@ -6,7 +6,7 @@ matrix:
           group: beta
           sudo: false
           env:
-            - CC=gcc SANITIZE_ARG=--sanitize
+            - CC=gcc CFLAGS=-funsigned-char SANITIZE_ARG=--sanitize
           language: python
           python: "3.5"
           addons:
@@ -20,8 +20,6 @@ matrix:
                 - libxcursor-dev
                 - libunistring-dev
                 - libxcb-xkb-dev
-                - libxkbcommon-dev
-                - libxkbcommon-x11-dev
                 - libpng-dev
                 - python3-pil
 
@@ -44,8 +42,6 @@ matrix:
                 - libxcursor-dev
                 - libunistring-dev
                 - libxcb-xkb-dev
-                - libxkbcommon-dev
-                - libxkbcommon-x11-dev
                 - libpng-dev
                 - python3-pil
 
@@ -68,8 +64,6 @@ matrix:
                 - libxcursor-dev
                 - libunistring-dev
                 - libxcb-xkb-dev
-                - libxkbcommon-dev
-                - libxkbcommon-x11-dev
                 - libpng-dev
                 - python3-pil
                 - clang
@@ -102,6 +96,12 @@ install: |
         fi
     else
         pushd /tmp
+        wget --no-check-certificate http://xkbcommon.org/download/libxkbcommon-0.6.1.tar.xz
+        tar xf libxkbcommon-0.6.1.tar.xz
+        cd libxkbcommon-0.6.1
+        ./configure --prefix=$HOME/harfbuzz --disable-dependency-tracking --disable-static
+        make -j2
+        make install
         wget https://github.com/behdad/harfbuzz/releases/download/1.6.3/harfbuzz-1.6.3.tar.bz2
         tar xf harfbuzz-1.6.3.tar.bz2
         cd harfbuzz-1.6.3

CHANGELOG.rst

@@ -3,6 +3,172 @@ Changelog
 
 kitty is a feature full, cross-platform, *fast*, GPU based terminal emulator.
 
+version 0.10.0 [2018-05-21]
+------------------------------
+
+- A diff kitten to show side-by-side diffs with syntax highlighting and support
+  for images. See `diff README
+  <https://github.com/kovidgoyal/kitty/tree/master/kittens/diff>`_.
+
+- Make windows in the various kitty layouts manually resizable. See the
+  `Layouts section of the README
+  <https://github.com/kovidgoyal/kitty#layouts>`_ for details.
+
+- Implement support for the SGR *faint* escape code to make text blend
+  into the background (`#446 <https://github.com/kovidgoyal/kitty/issues/446>`_).
+
+- Make the hints kitten a little `smarter
+  <https://github.com/kovidgoyal/kitty/commit/ad1109b6fe0a6802ca4f77182a7a0b36086b3e9f>`_,
+  so that URLs that stretch over multiple lines are detected. Also improve
+  detection of surrounding brackets/quotes.
+
+- Make the kitty window id available as the environment variable
+  `KITTY_WINDOW_ID` (`#532 <https://github.com/kovidgoyal/kitty/issues/532>`_).
+
+- Add a "fat" layout that is similar to the "tall" layout but vertically
+  oriented.
+
+- Expand environment variables in config file include directives
+
+- Allow programs running in kitty to `read/write from the clipboard
+  <https://github.com/kovidgoyal/kitty/commit/889ca7791244253cb08fbc3eca8883a87fb943a7>`_.
+  By default only writing is allowed.
+
+- Fix moving cursor outside a defined page area incorrectly causing the cursor
+  to be placed inside the page area. Caused incorrect rendering in neovim, in
+  some situations.
+
+- Render a couple more powerline symbols directly, bypassing the font
+  (`#550 <https://github.com/kovidgoyal/kitty/issues/550>`_).
+
+- Fix ctrl+alt+<special> not working in normal and application keyboard (`#548
+  <https://github.com/kovidgoyal/kitty/issues/548>`_).
+
+- Partial fix for rendering Right-to-left languages like Arabic. Rendering of
+  Arabic is never going to be perfect, but now it is at least readable.
+
+- Fix Ctrl+backspace acting as plain backspace in normal and application
+  keyboard modes (`#538 <https://github.com/kovidgoyal/kitty/issues/538>`_). 
+
+- Have the paste_from_selection action paste from the clipboard on platforms
+  that do not have a primary selection such as Wayland and macOS 
+  (`#529 <https://github.com/kovidgoyal/kitty/issues/529>`_)
+
+- Fix cursor_stop_blinking_after=0 not working (`#530 <https://github.com/kovidgoyal/kitty/issues/530>`_)
+
+
+version 0.9.1 [2018-05-05]
+------------------------------
+
+- Show a bell symbol on the tab if a bell occurs in one of the windows in the tab and
+  the window is not the currently focused window
+
+- Change the window border color if a bell occurs in an unfocused window. Can
+  be disabled by setting the bell_border_color to be the same as the
+  inactive_border_color.
+
+- macOS: Add support for dead keys
+
+- Unicode input: When searching by name search for prefix matches as well as
+  whole word matches
+
+- Dynamically allocate the memory used for the scrollback history buffer.
+  Reduces startup memory consumption when using very large scrollback
+  buffer sizes.
+
+- Add an option to not request window attention on bell.
+
+- Remote control: Allow matching windows by number (visible position).
+
+- macOS: Fix changing tab title and kitty shell not working
+
+- When triple-clicking select all wrapped lines belonging to a single logical line.
+
+- hints kitten: Detect bracketed URLs and dont include the closing bracket in the URL.
+
+- When calling pass_selection_to_program use the current directory of the child
+  process as the cwd of the program.
+
+- Add macos_hide_from_tasks option to hide kitty from the macOS task switcher
+
+- macOS: When the macos_titlebar_color is set to background change the titlebar
+  colors to match the current background color of the active kitty window
+
+- Add a setting to clear all shortcuts defined up to that point in the config
+  file(s)
+
+- Add a setting (kitty_mod) to change the modifier used by all the default
+  kitty shortcuts, globally
+
+- Fix Shift+function key not working
+
+- Support the F13 to F25 function keys
+  
+- Don't fail to start if the user deletes the hintstyle key from their
+  fontconfig configuration.
+
+- When rendering a private use unicode codepoint and a space as a two cell
+  ligature, set the foreground colors of the space cell to match the colors of
+  the first cell. Works around applications like powerline that use different
+  colors for the two cells.
+
+- Fix passing @text to other programs such as when viewing the scrollback
+  buffer not working correctly if kitty is itself scrolled up.
+
+- Fix window focus gained/lost events not being reported to child programs when
+  switching windows/tabs using the various keyboard shortcuts.
+
+- Fix tab title not changing to reflect the window title when switching between different windows in a tab
+
+- Ignore -e if it is specified on the command line. This is for compatibility
+  with broken software that assumes terminals should run with an -e option to
+  execute commands instead of just passing the commands as arguments. 
+
+
+version 0.9.0 [2018-04-15]
+------------------------------
+
+- A new kitty command shell to allow controlling kitty via commands. Press
+  `ctrl+shift+escape` to run the shell.
+
+- The hints kitten has become much more powerful. Now in addition to URLs you
+  can use it to select word, paths, filenames, lines, etc. from the screen.
+  These can be inserted into the terminal, copied to clipboard or sent to
+  external programs.
+
+- Linux: Switch to libxkbcommon for keyboard handling. It allows kitty to
+  support XCompose and dead keys and also react to keyboard remapping/layout
+  change without needing a restart.
+
+- Add support for multiple-key-sequence shortcuts
+
+- A new remote control command `set-colors` to change the current and/or
+  configured colors.
+
+- When double-clicking to select a word, select words that continue onto the
+  next/prev line as well.
+
+- Add an `include` directive for the config files to read multiple config files
+
+- Improve mouse selection for windows with padding. Moving the mouse into the
+  padding area now acts as if the mouse is over the nearest cell.
+
+- Allow setting all 256 terminal colors in the config file
+
+- Fix using `kitty --single-instance` to open a new window in a running kitty
+  instance, not respecting the `--directory` flag
+
+- URL hints: Exclude trailing punctuation from URLs
+
+- URL hints: Launch the browser from the kitty parent process rather than the
+  hints kitten. Fixes launching on some systems where xdg-open doesn't like
+  being run from a kitten.
+
+- Allow using rectangle select mode by pressing shift in addition to the
+  rectangle select modifiers even when the terminal program has grabbed the
+  mouse.
+
+
 version 0.8.4 [2018-03-31]
 -----------------------------
 

README.asciidoc

@@ -13,6 +13,7 @@
 :sc_fourth_window: pass:quotes[`ctrl+shift+4`]
 :sc_increase_font_size: pass:quotes[`ctrl+shift+equal`]
 :sc_input_unicode_character: pass:quotes[`ctrl+shift+u`]
+:sc_kitty_shell_window: pass:quotes[`ctrl+shift+escape`]
 :sc_move_tab_backward: pass:quotes[`ctrl+shift+,`]
 :sc_move_tab_forward: pass:quotes[`ctrl+shift+.`]
 :sc_move_window_backward: pass:quotes[`ctrl+shift+b`]
@@ -31,7 +32,11 @@
 :sc_previous_tab: pass:quotes[`ctrl+shift+left`]
 :sc_previous_window: pass:quotes[`ctrl+shift+[`]
 :sc_restore_font_size: pass:quotes[`ctrl+shift+backspace`]
-:sc_run_simple_kitten_text_url_hints: pass:quotes[`ctrl+shift+e`]
+:sc_run_kitten_text_hints: pass:quotes[`ctrl+shift+e`]
+:sc_run_kitten_text_hints_type_line_program: pass:quotes[`ctrl+shift+p → l`]
+:sc_run_kitten_text_hints_type_path: pass:quotes[`ctrl+shift+p → shift+f`]
+:sc_run_kitten_text_hints_type_path_program: pass:quotes[`ctrl+shift+p → f`]
+:sc_run_kitten_text_hints_type_word_program: pass:quotes[`ctrl+shift+p → w`]
 :sc_scroll_end: pass:quotes[`ctrl+shift+end`]
 :sc_scroll_home: pass:quotes[`ctrl+shift+home`]
 :sc_scroll_line_down: pass:quotes[`ctrl+shift+down` or `ctrl+shift+j`]
@@ -43,6 +48,7 @@
 :sc_seventh_window: pass:quotes[`ctrl+shift+7`]
 :sc_show_scrollback: pass:quotes[`ctrl+shift+h`]
 :sc_sixth_window: pass:quotes[`ctrl+shift+6`]
+:sc_start_resizing_window: pass:quotes[`ctrl+shift+r`]
 :sc_tenth_window: pass:quotes[`ctrl+shift+0`]
 :sc_third_window: pass:quotes[`ctrl+shift+3`]
 :sc_toggle_fullscreen: pass:quotes[`ctrl+shift+f11`]
@@ -68,7 +74,7 @@ link:#layouts[layouts] without needing to use an extra program like tmux
 * Can be link:remote-control.asciidoc[controlled from scripts or the shell prompt], even over SSH.
 
 * Has a framework for _kittens_, small terminal programs that can be used to extend kitty's functionality.
-For example, they are used for link:#unicode-input[Unicode input] and link:#url-hints[URL hints].
+For example, they are used for link:#unicode-input[Unicode input] and link:#hints[Hints].
 
 * Supports link:#startup-sessions[startup sessions] which allow you to specify the window/tab layout,
 working directories and programs to run on startup.
@@ -82,7 +88,7 @@ window using arbitrary programs of your choice. This is useful for browsing
 the history comfortably in a pager or editor.
 
 
-image::screenshot.png?raw=true[Screenshot, showing three programs in the "Tall" layout]
+image::screenshots/screenshot.png?raw=true[Screenshot, showing three programs in the "Tall" layout]
 
 toc::[]
 
@@ -101,9 +107,10 @@ the following dependencies are installed first.
 * freetype (not needed on macOS)
 * fontconfig (not needed on macOS)
 * ImageMagick (optional, needed to use the `kitty icat` tool to display images in the terminal)
+* pygments (optional, need for syntax highlighting in `kitty +kitten diff`)
 * gcc or clang (required only for building)
 * pkg-config (required only for building)
-* For building on Linux in addition to the above dependencies you might also need to install the `-dev` packages for `xcursor`, `xrandr`, `xinerama`, `libgl1-mesa` and `xkbcommon-x11`, if they are not already installed by your distro.
+* For building on Linux in addition to the above dependencies you might also need to install the `-dev` packages for `xcursor`, `xrandr`, `libxi`, `xinerama`, `libgl1-mesa` and `xkbcommon-x11`, if they are not already installed by your distro.
 
 === Install and run from source
 
@@ -111,7 +118,7 @@ the following dependencies are installed first.
 git clone https://github.com/kovidgoyal/kitty && cd kitty
 ....
 
-Now build the C parts of kitty with the following command:
+Now build the native code parts of kitty with the following command:
 
 ....
 make
@@ -123,11 +130,23 @@ You can run kitty, as:
 python3 .
 ....
 
+If that works, you can create a script to launch kitty:
+
+....
+#!/usr/bin/env python3
+
+import runpy
+runpy.run_path('/path/to/kitty/dir', run_name='__main__')
+....
+
+And place it in `~/bin` or `/usr/bin` so that you can run kitty using
+just `kitty`.
+
 === Linux packages
 
-* Arch Linux: AUR git package https://aur.archlinux.org/packages/kitty-git/
+* Arch Linux: https://www.archlinux.org/packages/community/x86_64/kitty/
 
-* Arch Linux: AUR stable package https://aur.archlinux.org/packages/kitty/
+* Arch Linux: AUR git package https://aur.archlinux.org/packages/kitty-git/
 
 * NixOS / nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/misc/kitty/default.nix
 
@@ -135,6 +154,8 @@ python3 .
 
 * openSUSE: https://build.opensuse.org/package/show/X11:terminals/kitty
 
+* Debian: https://packages.qa.debian.org/k/kitty/news/20180419T070013Z.html
+
 === macOS packages
 
 kitty is available as a macOS `dmg` file for easy installation from the
@@ -239,7 +260,8 @@ windows are:
 |Input unicode character | {sc_input_unicode_character}
 |Click URL using the keyboard | {sc_run_simple_kitten_text_url_hints}
 |Pass current selection to program | {sc_pass_selection_to_program}
-|Edit kitty config file| {sc_edit_config_file}
+|Edit kitty config file | {sc_edit_config_file}
+|Open a kitty shell | {sc_kitty_shell}
 |===
 
 
@@ -272,7 +294,7 @@ the scrollback pager comfortably within the pager.
 You can input unicode characters by name, hex code, recently used and even an editable favorites list.
 Press {sc_input_unicode_character} to start the unicode input widget, shown below.
 
-image::unicode.png?raw=true[Unicode input widget, showing selection of unicode characters by name]
+image::screenshots/unicode.png?raw=true[Unicode input widget, showing selection of unicode characters by name]
 
 In Code mode, you enter a unicode character by typing in the hex code for the
 character and pressing enter, for example, type in 2716 and press enter to get
@@ -284,12 +306,22 @@ keys/tab to select the character from the displayed matches. You can also type
 a leading period and the index for the match if you dont like to use arrow
 keys.
 
-== URL hints
+== Hints
 
-kitty has a _hints mode_ to click URLs visible on the screen by using only the keyboard.
-Press {sc_run_simple_kitten_text_url_hints} to activate hints mode, as shown below.
+kitty has a _hints mode_ to select and act on arbitrary text snippets currently
+visible on the screen.  For example, you can press {sc_run_kitten_text_hints}
+to choose any URL visible on the screen and then open it using your system
+browser.
 
-image::hints_mode.png?raw=true[URL hints mode]
+image::screenshots/hints_mode.png?raw=true[URL hints mode]
+
+Similarly, you can press {sc_run_kitten_text_hints_type_path_program} to
+select anything that looks like a path or filename and then insert it into the
+terminal, very useful for picking files from the output of a git or ls command and
+adding them to the command line for the next command.
+
+The hints kitten is very powerful to see more detailed help on its various
+options and modes of operation, use: `kitty +kitten hints --help`.
 
 == Miscellaneous features
 
@@ -304,6 +336,7 @@ Currently, there are five layouts available,
 
 * Stack -- Only a single maximized window is shown at a time
 * Tall -- One window is shown full height on the left, the rest of the windows are shown one below the other on the right
+* Fat -- One window is shown full width on the top, the rest of the windows are shown side-by-side on the bottom
 * Grid -- All windows are shown in a grid
 * Horizontal -- All windows are shown side-by-side
 * Vertical -- All windows are shown one below the other
@@ -312,7 +345,25 @@ You can switch between layouts using the {sc_next_layout} key combination. You c
 also create shortcuts to select particular layouts, and choose which layouts
 you want to enable/disable, see link:kitty/kitty.conf[kitty.conf] for examples.
 
-Writing a new layout only requires about fifty lines of code, so if there is
+You can resize windows inside layouts. Press {sc_start_resizing_window} to
+enter resizing mode and follow the on-screen instructions.  In a given window
+layout only some operations may be possible for a particular window. For
+example, in the Tall layout you can make the first window wider/narrower, but
+not taller/shorter. Note that what you are resizing is actually not a window,
+but a row/column in the layout, all windows in that row/column will be resized.
+
+Some layouts take options to control their behavior. For example, the `fat` and `tall`
+layouts accept the `bias` option to control how the available space is split up. To specify the
+option, in kitty.conf use:
+
+```
+enabled_layouts tall:bias=70
+```
+
+This will make the tall window occupy `70%` of available width. `bias` can be
+any number between 10 and 90.
+
+Writing a new layout only requires about a hundred lines of code, so if there is
 some layout you want, take a look at link:kitty/layout.py[layout.py] and submit
 a pull request!
 
@@ -328,7 +379,8 @@ By default kitty looks for a config file in the OS config directories (usually
 `~/Library/Preferences/kitty/kitty.conf` on macOS) but you can pass a specific
 path via the `--config` option or use the `KITTY_CONFIG_DIRECTORY` environment
 variable. See the help for the `--config` option in `kitty --help` for full
-details.
+details. You can also dump the current configuration using the `--debug-config`
+option.
 
 
 == Startup Sessions
@@ -512,21 +564,38 @@ Either install kitty on the server, or simply copy over the terminfo files,
 using:
 
 ....
-ssh myserver mkdir -p \~/.terminfo
-infocmp xterm-kitty | ssh myserver "cat > /tmp/kitty.terminfo"
-ssh myserver tic -x -o \~/.terminfo /tmp/kitty.terminfo
+infocmp xterm-kitty | ssh myserver tic -x -o \~/.terminfo /dev/stdin
 ....
 
 Really, the correct solution for this is to convince the OpenSSH maintainers to
 have ssh do this automatically when connecting to a server, so that all
 terminals work transparently.
 
+In the meantime, you can automate it by using a simple script called, for example,
+`~/bin/myssh`
+
+....
+#!/bin/sh
+infocmp xterm-kitty | ssh $1 tic -x -o \~/.terminfo /dev/stdin && ssh $1
+....
+
+Then you can use `myssh server` to log in to `server` with the terminfo file
+being automatically available.
+
+
+=== How do I change the colors in a running kitty instance?
+
+You can either use the
+link:http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Operating-System-Commands[OSC
+terminal escape codes] to set colors or you can enable link:remote-control.asciidoc[remote control]
+for kitty and use `kitty @ set-colors --help`.
+
 
 === How do I specify command line options for kitty on macOS?
 
 Apple does not want you to use command line options with GUI applications. To
 workaround that limitation, kitty will read command line options from the file
-`~/Library/Preferences/kitty/macos-launch-services-cmdline` when it is launched
+`<kitty config dir>/macos-launch-services-cmdline` when it is launched
 from the GUI, i.e. by clicking the kitty application icon or using `open -a kitty`.
 Note that this file is *only read* when running via the GUI.
 
@@ -536,6 +605,20 @@ You can, of course, also run kitty from a terminal with command line options, us
 And within kitty itself, you can always run kitty using just `kitty` as it
 cleverly adds itself to the PATH.
 
+
+=== kitty is not able to use my favorite font?
+
+kitty achieves its stellar performance by caching alpha masks of each rendered
+character on the GPU, so that every character needs to be rendered only once.
+This means it is a strictly character cell based display.  As such it can use
+only monospace fonts, since every cell in the grid has to be the same size. If
+your font is not listed in `kitty list-fonts` it means that it is not
+monospace. On Linux you can list all monospace fonts with,
+
+```
+fc-list : family spacing | grep spacing=100
+```
+
 == Resources on terminal behavior
 
 http://invisible-island.net/xterm/ctlseqs/ctlseqs.html

__main__.py

@@ -6,7 +6,7 @@ import sys
 
 
 def icat(args):
-    from kitty.icat import main
+    from kittens.icat.main import main
     main(args)
 
 
@@ -31,6 +31,13 @@ def launch(args):
     runpy.run_path(args[1], run_name='__main__')
 
 
+def run_kitten(args):
+    kitten = args[1]
+    sys.argv = args[1:]
+    from kittens.runner import run_kitten
+    run_kitten(kitten)
+
+
 def namespaced(args):
     func = namespaced_entry_points[args[1]]
     func(args[1:])
@@ -42,6 +49,7 @@ entry_points = {
     'list-fonts': list_fonts,
     'runpy': runpy,
     'launch': launch,
+    'kitten': run_kitten,
 
     '@': remote_control,
     '+': namespaced,

gen-wcwidth.py

@@ -44,7 +44,8 @@ def parse_ucd():
     def add_word(w, c):
         if c <= 32 or c == 127 or 128 <= c <= 159:
             return
-        word_search_map[w.lower()].add(c)
+        if len(w) > 1:
+            word_search_map[w.lower()].add(c)
 
     first = None
     for word, c in html5.items():
@@ -54,7 +55,9 @@ def parse_ucd():
     for line in get_data('ucd/UnicodeData.txt'):
         parts = [x.strip() for x in line.split(';')]
         codepoint = int(parts[0], 16)
-        name = parts[1]
+        name = parts[1] or parts[10]
+        if name == '<control>':
+            name = parts[10]
         if name:
             name_map[codepoint] = name
             for word in name.lower().split():
@@ -207,7 +210,8 @@ def codepoint_to_mark_map(p, mark_map):
     for spec in get_ranges(mark_map):
         if isinstance(spec, tuple):
             s = rmap[spec[0]]
-            p(f'\t\tcase {spec[0]} ... {spec[1]}: return {s} + c - {spec[0]};')
+            cases = ' '.join(f'case {i}:' for i in range(spec[0], spec[1]+1))
+            p(f'\t\t{cases} return {s} + c - {spec[0]};')
         else:
             p(f'\t\tcase {spec}: return {rmap[spec]};')
     p('default: return 0;')
@@ -215,10 +219,12 @@ def codepoint_to_mark_map(p, mark_map):
     return rmap
 
 
-def classes_to_regex(classes):
+def classes_to_regex(classes, exclude=''):
     chars = set()
     for c in classes:
         chars |= class_maps[c]
+    for c in map(ord, exclude):
+        chars.discard(c)
 
     def as_string(codepoint):
         if codepoint < 256:
@@ -256,53 +262,111 @@ def gen_ucd():
             raise ValueError('The mark for 0xfe0e has changed, you have to update VS15 to {} and VS16 to {} in unicode-data.h'.format(
                 rmap[0xfe0e], rmap[0xfe0f]
             ))
-    with open('kittens/url_hints/url_regex.py', 'w') as f:
-        f.write("url_delimiters = '{}'  # noqa".format(''.join(classes_to_regex(cz))))
+    with open('kittens/hints/url_regex.py', 'w') as f:
+        f.write("url_delimiters = '{}'  # noqa".format(''.join(classes_to_regex(cz, exclude='\n'))))
 
 
 def gen_names():
-    words = tuple(sorted(word_search_map))
-
     with create_header('kittens/unicode_input/names.h') as p:
-        cp_map = list(sorted(name_map))
-        p(f'static const char* name_map[{len(cp_map)}] = {{' ' // {{{')
-        for cp in cp_map:
+        mark_to_cp = list(sorted(name_map))
+        cp_to_mark = {cp: m for m, cp in enumerate(mark_to_cp)}
+        # Mapping of mark to codepoint name
+        p(f'static const char* name_map[{len(mark_to_cp)}] = {{' ' // {{{')
+        for cp in mark_to_cp:
             w = name_map[cp].replace('"', '\\"')
             p(f'\t"{w}",')
         p("}; // }}}\n")
 
-        p(f'static const char* idx_to_word[{len(words)}] = ' '{ // {{{')
-        for s in words:
-            s = s.replace('"', '\\"')
-            p(f'\t"{s}",')
-        p("}; // }}}\n")
-
-        first_letters = {ord(w[0]) for w in words if ord(w[0]) < 256}
-        wmap = {w: i for i, w in enumerate(words)}
-        p(f'static const unsigned short* words_for_first_letter[256] = ' '{ // {{{')
-        for fl in range(0, 256):
-            if fl in first_letters:
-                winds = [str(wmap[w]) for w in words if w.startswith(chr(fl))]
-                p(f'\t(const unsigned short[{len(winds) + 1}]){{{len(winds)}, ', ', '.join(winds), '},')
-            else:
-                p('NULL,')
-        p("}; // }}}\n")
-
-        p(f'static const char_type* codepoints_for_word_idx[{len(words)}] = ' '{ // {{{')
-        for s in words:
-            cps = word_search_map[s]
-            a = ', '.join(map(str, cps))
-            p(f'\t(const char_type[{len(cps) + 1}]){{{len(cps)}, ', a, '},')
-        p("}; // }}}\n")
+        # Mapping of mark to codepoint
+        p(f'static const char_type mark_to_cp[{len(mark_to_cp)}] = {{' ' // {{{')
+        p(', '.join(map(str, mark_to_cp)))
+        p('}; // }}}\n')
 
+        # Function to get mark number for codepoint
         p('static char_type mark_for_codepoint(char_type c) {')
-        codepoint_to_mark_map(p, cp_map)
+        codepoint_to_mark_map(p, mark_to_cp)
         p('}\n')
         p('static inline const char* name_for_codepoint(char_type cp) {')
         p('\tchar_type m = mark_for_codepoint(cp); if (m == 0) return NULL;')
         p('\treturn name_map[m];')
         p('}\n')
 
+        # Array of all words
+        word_map = tuple(sorted(word_search_map))
+        word_rmap = {w: i for i, w in enumerate(word_map)}
+        p(f'static const char* all_words_map[{len(word_map)}] = {{' ' // {{{')
+        cwords = (w.replace('"', '\\"') for w in word_map)
+        p(', '.join(f'"{w}"' for w in cwords))
+        p('}; // }}}\n')
+
+        # Array of sets of marks for each word
+        word_to_marks = {word_rmap[w]: frozenset(map(cp_to_mark.__getitem__, cps)) for w, cps in word_search_map.items()}
+        all_mark_groups = frozenset(word_to_marks.values())
+        array = [0]
+        mg_to_offset = {}
+        for mg in all_mark_groups:
+            mg_to_offset[mg] = len(array)
+            array.append(len(mg))
+            array.extend(sorted(mg))
+        p(f'static const char_type mark_groups[{len(array)}] = {{' ' // {{{')
+        p(', '.join(map(str, array)))
+        p('}; // }}}\n')
+        offsets_array = []
+        for wi, w in enumerate(word_map):
+            mg = word_to_marks[wi]
+            offsets_array.append(mg_to_offset[mg])
+        p(f'static const char_type mark_to_offset[{len(offsets_array)}] = {{' ' // {{{')
+        p(', '.join(map(str, offsets_array)))
+        p('}; // }}}\n')
+
+        # The trie
+        p(f'typedef struct {{ uint32_t children_offset; uint32_t match_offset; }} word_trie;\n')
+        all_trie_nodes = []
+
+        class TrieNode:
+
+            def __init__(self):
+                self.match_offset = 0
+                self.children_offset = 0
+                self.children = {}
+
+            def add_letter(self, letter):
+                if letter not in self.children:
+                    self.children[letter] = len(all_trie_nodes)
+                    all_trie_nodes.append(TrieNode())
+                return self.children[letter]
+
+            def __str__(self):
+                return f'{{ .children_offset={self.children_offset}, .match_offset={self.match_offset} }}'
+
+        root = TrieNode()
+        all_trie_nodes.append(root)
+
+        def add_word(word_idx):
+            word = word_map[word_idx]
+            parent = root
+            for letter in map(ord, word):
+                idx = parent.add_letter(letter)
+                parent = all_trie_nodes[idx]
+            parent.match_offset = offsets_array[word_idx]
+
+        for i in range(len(word_map)):
+            add_word(i)
+        children_array = [0]
+        for node in all_trie_nodes:
+            if node.children:
+                node.children_offset = len(children_array)
+                children_array.append(len(node.children))
+                for letter, child_offset in node.children.items():
+                    children_array.append((child_offset << 8) | (letter & 0xff))
+
+        p(f'static const word_trie all_trie_nodes[{len(all_trie_nodes)}] = {{' ' // {{{')
+        p(',\n'.join(map(str, all_trie_nodes)))
+        p('\n}; // }}}\n')
+        p(f'static const uint32_t children_array[{len(children_array)}] = {{' ' // {{{')
+        p(', '.join(map(str, children_array)))
+        p('}; // }}}\n')
+
 
 def gen_wcwidth():
     seen = set()

glad/generate.py

@@ -27,6 +27,10 @@ def regenerate():
     )
 
 
+def strip_trailing_whitespace(c):
+    return re.sub(r'\s+$', '', c, flags=re.MULTILINE) + '\n'
+
+
 def export():
     c = open('out/src/glad.c', 'rb').read().decode('utf-8')
     functions = []
@@ -48,12 +52,13 @@ init_glad(GLADloadproc load, int debug) {
     }
     return ret;
 }'''.replace('SUB', ';\n        '.join(switch), 1)
-    open('../kitty/gl-wrapper.c', 'w').write(c)
+    open('../kitty/gl-wrapper.c', 'w').write(strip_trailing_whitespace(c))
     raw = open('out/include/glad/glad.h').read()
     raw = raw.replace('<KHR/khrplatform.h>', '"khrplatform.h"')
     raw += '\nint init_glad(GLADloadproc, int);\n'
-    open('../kitty/gl-wrapper.h', 'w').write(raw)
-    shutil.copy2('out/include/KHR/khrplatform.h', '../kitty')
+    open('../kitty/gl-wrapper.h', 'w').write(strip_trailing_whitespace(raw))
+    raw = open('out/include/KHR/khrplatform.h', 'rb').read().decode('utf-8')
+    open('../kitty/khrplatform.h', 'w').write(strip_trailing_whitespace(raw))
 
 
 if __name__ == '__main__':

glfw/cocoa_init.m

@@ -69,6 +69,7 @@ static void changeToResourcesDirectory(void)
 static void createKeyTables(void)
 {
     int scancode;
+    _glfw.ns.debug_keyboard = getenv("GLFW_DEBUG_KEYBOARD") != NULL;
 
     memset(_glfw.ns.keycodes, -1, sizeof(_glfw.ns.keycodes));
     memset(_glfw.ns.scancodes, -1, sizeof(_glfw.ns.scancodes));
@@ -207,6 +208,9 @@ static GLFWbool updateUnicodeDataNS(void)
         _glfw.ns.unicodeData = nil;
     }
 
+    for (_GLFWwindow *window = _glfw.windowListHead;  window;  window = window->next)
+        window->ns.deadKeyState = 0;
+
     _glfw.ns.inputSource = TISCopyCurrentKeyboardLayoutInputSource();
     if (!_glfw.ns.inputSource)
     {
@@ -371,4 +375,3 @@ const char* _glfwPlatformGetVersionString(void)
 #endif
         ;
 }
-

glfw/cocoa_platform.h

@@ -37,6 +37,7 @@ typedef void* id;
 #endif
 
 typedef VkFlags VkMacOSSurfaceCreateFlagsMVK;
+typedef int (* GLFWcocoatextinputfilterfun)(int,int,int);
 
 typedef struct VkMacOSSurfaceCreateInfoMVK
 {
@@ -98,6 +99,10 @@ typedef struct _GLFWwindowNS
     // This is kept to counteract Cocoa doing the same internally
     double          cursorWarpDeltaX, cursorWarpDeltaY;
 
+    // The text input filter callback
+    GLFWcocoatextinputfilterfun textInputFilterCallback;
+    // Dead key state
+    UInt32 deadKeyState;
 } _GLFWwindowNS;
 
 // Cocoa-specific global data
@@ -114,6 +119,8 @@ typedef struct _GLFWlibraryNS
     id                  listener;
 
     char                keyName[64];
+    char                text[256];
+    GLFWbool            debug_keyboard;
     short int           keycodes[256];
     short int           scancodes[GLFW_KEY_LAST + 1];
     char*               clipboardString;
@@ -166,4 +173,3 @@ void _glfwInitTimerNS(void);
 void _glfwPollMonitorsNS(void);
 void _glfwSetVideoModeNS(_GLFWmonitor* monitor, const GLFWvidmode* desired);
 void _glfwRestoreVideoModeNS(_GLFWmonitor* monitor);
-

glfw/cocoa_window.m

@@ -128,6 +128,32 @@ static void updateCursorImage(_GLFWwindow* window)
         hideCursor(window);
 }
 
+// Apply chosen cursor mode to a focused window
+//
+static void updateCursorMode(_GLFWwindow* window)
+{
+    if (window->cursorMode == GLFW_CURSOR_DISABLED)
+    {
+        _glfw.ns.disabledCursorWindow = window;
+        _glfwPlatformGetCursorPos(window,
+                                  &_glfw.ns.restoreCursorPosX,
+                                  &_glfw.ns.restoreCursorPosY);
+        centerCursor(window);
+        CGAssociateMouseAndMouseCursorPosition(false);
+    }
+    else if (_glfw.ns.disabledCursorWindow == window)
+    {
+        _glfw.ns.disabledCursorWindow = NULL;
+        CGAssociateMouseAndMouseCursorPosition(true);
+        _glfwPlatformSetCursorPos(window,
+                                  _glfw.ns.restoreCursorPosX,
+                                  _glfw.ns.restoreCursorPosY);
+    }
+
+    if (cursorInClientArea(window))
+        updateCursorImage(window);
+}
+
 // Transforms the specified y-coordinate between the CG display and NS screen
 // coordinate systems
 //
@@ -183,10 +209,112 @@ static int translateFlags(NSUInteger flags)
     return mods;
 }
 
+#define debug_key(...) if (_glfw.ns.debug_keyboard) NSLog(__VA_ARGS__)
+
+static inline const char*
+format_mods(int mods) {
+    static char buf[128];
+    char *p = buf, *s;
+#define pr(x) p += snprintf(p, sizeof(buf) - (p - buf) - 1, x)
+    pr("mods: ");
+    s = p;
+    if (mods & GLFW_MOD_CONTROL) pr("ctrl+");
+    if (mods & GLFW_MOD_ALT) pr("alt+");
+    if (mods & GLFW_MOD_SHIFT) pr("shift+");
+    if (mods & GLFW_MOD_SUPER) pr("super+");
+    if (mods & GLFW_MOD_CAPS_LOCK) pr("capslock+");
+    if (mods & GLFW_MOD_NUM_LOCK) pr("numlock+");
+    if (p == s) pr("none");
+    else p--;
+    pr(" ");
+#undef pr
+    return buf;
+}
+
+static inline const char*
+format_text(const char *src) {
+    static char buf[256];
+    char *p = buf;
+    if (!src[0]) return "<none>";
+    while (*src) {
+        p += snprintf(p, sizeof(buf) - (p - buf), "%x ", (unsigned char)*(src++));
+    }
+    if (p != buf) *(--p) = 0;
+    return buf;
+}
+
+static const char*
+safe_name_for_scancode(unsigned int scancode) {
+    const char *ans = _glfwPlatformGetScancodeName(scancode);
+    if (!ans) return "<noname>";
+    if ((1 <= ans[0] && ans[0] <= 31) || ans[0] == 127) ans = "<cc>";
+    return ans;
+}
+
+
 // Translates a macOS keycode to a GLFW keycode
 //
-static int translateKey(unsigned int key)
+static int translateKey(unsigned int key, GLFWbool apply_keymap)
 {
+    if (apply_keymap) {
+        // Look for the effective key name after applying any keyboard layouts/mappings
+        const char *name = _glfwPlatformGetScancodeName(key);
+        if (name && name[1] == 0) {
+            // Single letter key name
+            switch(name[0]) {
+#define K(ch, name) case ch: return GLFW_KEY_##name
+                K('A', A); K('a', A);
+                K('B', B); K('b', B);
+                K('C', C); K('c', C);
+                K('D', D); K('d', D);
+                K('E', E); K('e', E);
+                K('F', F); K('f', F);
+                K('G', G); K('g', G);
+                K('H', H); K('h', H);
+                K('I', I); K('i', I);
+                K('J', J); K('j', J);
+                K('K', K); K('k', K);
+                K('L', L); K('l', L);
+                K('M', M); K('m', M);
+                K('N', N); K('n', N);
+                K('O', O); K('o', O);
+                K('P', P); K('p', P);
+                K('Q', Q); K('q', Q);
+                K('R', R); K('r', R);
+                K('S', S); K('s', S);
+                K('T', T); K('t', T);
+                K('U', U); K('u', U);
+                K('V', V); K('v', V);
+                K('W', W); K('w', W);
+                K('X', X); K('x', X);
+                K('Y', Y); K('y', Y);
+                K('Z', Z); K('z', Z);
+                K('0', 0);
+                K('1', 1);
+                K('2', 2);
+                K('3', 3);
+                K('5', 5);
+                K('6', 6);
+                K('7', 7);
+                K('8', 8);
+                K('9', 9);
+                K('\'', APOSTROPHE);
+                K(',', COMMA);
+                K('.', PERIOD);
+                K('/', SLASH);
+                K('-', MINUS);
+                K('=', EQUAL);
+                K(';', SEMICOLON);
+                K('[', LEFT_BRACKET);
+                K(']', RIGHT_BRACKET);
+                K('`', GRAVE_ACCENT);
+                K('\\', BACKSLASH);
+#undef K
+                default:
+                    break;
+            }
+        }
+    }
     if (key >= sizeof(_glfw.ns.keycodes) / sizeof(_glfw.ns.keycodes[0]))
         return GLFW_KEY_UNKNOWN;
 
@@ -321,7 +449,7 @@ static const NSRange kEmptyRange = { NSNotFound, 0 };
         centerCursor(window);
 
     _glfwInputWindowFocus(window, GLFW_TRUE);
-    _glfwPlatformSetCursorMode(window, window->cursorMode);
+    updateCursorMode(window);
 }
 
 - (void)windowDidResignKey:(NSNotification *)notification
@@ -617,14 +745,99 @@ static const NSRange kEmptyRange = { NSNotFound, 0 };
     [super updateTrackingAreas];
 }
 
+static inline UInt32
+convert_cocoa_to_carbon_modifiers(NSUInteger flags) {
+    UInt32 mods = 0;
+    if (flags & NSEventModifierFlagShift)
+        mods |= shiftKey;
+    if (flags & NSEventModifierFlagControl)
+        mods |= controlKey;
+    if (flags & NSEventModifierFlagOption)
+        mods |= optionKey;
+    if (flags & NSEventModifierFlagCommand)
+        mods |= cmdKey;
+    if (flags & NSEventModifierFlagCapsLock)
+        mods |= alphaLock;
+
+    return (mods >> 8) & 0xFF;
+}
+
+static inline void
+convert_utf16_to_utf8(UniChar *src, UniCharCount src_length, char *dest, size_t dest_sz) {
+    CFStringRef string = CFStringCreateWithCharactersNoCopy(kCFAllocatorDefault,
+                                                            src,
+                                                            src_length,
+                                                            kCFAllocatorNull);
+    CFStringGetCString(string,
+                       dest,
+                       dest_sz,
+                       kCFStringEncodingUTF8);
+    CFRelease(string);
+}
+
+static inline GLFWbool
+is_ascii_control_char(char x) {
+    return x == 0 || (1 <= x && x <= 31) || x == 127;
+}
+
 - (void)keyDown:(NSEvent *)event
 {
-    const int key = translateKey([event keyCode]);
-    const int mods = translateFlags([event modifierFlags]);
-
-    _glfwInputKey(window, key, [event keyCode], GLFW_PRESS, mods);
-
-    [self interpretKeyEvents:[NSArray arrayWithObject:event]];
+    const unsigned int scancode = [event keyCode];
+    const NSUInteger flags = [event modifierFlags];
+    const int mods = translateFlags(flags);
+    const int key = translateKey(scancode, GLFW_TRUE);
+    const GLFWbool process_text = !window->ns.textInputFilterCallback || window->ns.textInputFilterCallback(key, mods, scancode) != 1;
+    _glfw.ns.text[0] = 0;
+    if (!_glfw.ns.unicodeData) {
+        // Using the cocoa API for key handling is disabled, as there is no
+        // reliable way to handle dead keys using it. Only use it if the
+        // keyboard unicode data is not available.
+        if (process_text) {
+            // this will call insertText with the text for this event, if any
+            [self interpretKeyEvents:[NSArray arrayWithObject:event]];
+        }
+    } else {
+        static UniChar text[256];
+        UniCharCount char_count = 0;
+        if (UCKeyTranslate(
+                    [(NSData*) _glfw.ns.unicodeData bytes],
+                    scancode,
+                    kUCKeyActionDown,
+                    convert_cocoa_to_carbon_modifiers(flags),
+                    LMGetKbdType(),
+                    (process_text ? 0 : kUCKeyTranslateNoDeadKeysMask),
+                    &(window->ns.deadKeyState),
+                    sizeof(text)/sizeof(text[0]),
+                    &char_count,
+                    text
+                    ) != noErr) {
+            debug_key(@"UCKeyTranslate failed for scancode: 0x%x (%s) %s\n", scancode, safe_name_for_scancode(scancode), format_mods(mods));
+            window->ns.deadKeyState = 0;
+            return;
+        }
+        if (process_text) {
+            // We check if cocoa wants to insert text, as UCKeyTranslate
+            // inserts text even when the cmd key is pressed. For instance,
+            // cmd+a will result in the text a.
+            [self interpretKeyEvents:[NSArray arrayWithObject:event]];
+            debug_key(@"char_count: %lu cocoa text: %s\n", char_count, format_text(_glfw.ns.text));
+            GLFWbool cocoa_wants_to_insert_text = !is_ascii_control_char(_glfw.ns.text[0]);
+            _glfw.ns.text[0] = 0;
+            if (char_count && cocoa_wants_to_insert_text) convert_utf16_to_utf8(text, char_count, _glfw.ns.text, sizeof(_glfw.ns.text));
+        } else {
+            window->ns.deadKeyState = 0;
+        }
+        if (window->ns.deadKeyState && char_count == 0) {
+            debug_key(@"Ignoring dead key. deadKeyState: 0x%x scancode: 0x%x (%s) %s\n",
+                    window->ns.deadKeyState, scancode, safe_name_for_scancode(scancode), format_mods(mods));
+            return;
+        }
+    }
+    debug_key(@"scancode: 0x%x (%s) %stext: %s glfw_key: %s\n",
+            scancode, safe_name_for_scancode(scancode), format_mods(mods),
+            format_text(_glfw.ns.text), _glfwGetKeyName(key));
+    if (is_ascii_control_char(_glfw.ns.text[0])) _glfw.ns.text[0] = 0;  // dont send text for ascii control codes
+    _glfwInputKeyboard(window, key, scancode, GLFW_PRESS, mods, _glfw.ns.text, 0);
 }
 
 - (void)flagsChanged:(NSEvent *)event
@@ -632,7 +845,7 @@ static const NSRange kEmptyRange = { NSNotFound, 0 };
     int action;
     const unsigned int modifierFlags =
         [event modifierFlags] & NSEventModifierFlagDeviceIndependentFlagsMask;
-    const int key = translateKey([event keyCode]);
+    const int key = translateKey([event keyCode], GLFW_FALSE);
     const int mods = translateFlags(modifierFlags);
     const NSUInteger keyFlag = translateKeyToModifierFlag(key);
 
@@ -646,14 +859,14 @@ static const NSRange kEmptyRange = { NSNotFound, 0 };
     else
         action = GLFW_RELEASE;
 
-    _glfwInputKey(window, key, [event keyCode], action, mods);
+    _glfwInputKeyboard(window, key, [event keyCode], action, mods, "", 0);
 }
 
 - (void)keyUp:(NSEvent *)event
 {
-    const int key = translateKey([event keyCode]);
+    const int key = translateKey([event keyCode], GLFW_TRUE);
     const int mods = translateFlags([event modifierFlags]);
-    _glfwInputKey(window, key, [event keyCode], GLFW_RELEASE, mods);
+    _glfwInputKeyboard(window, key, [event keyCode], GLFW_RELEASE, mods, "", 0);
 }
 
 - (void)scrollWheel:(NSEvent *)event
@@ -788,25 +1001,12 @@ static const NSRange kEmptyRange = { NSNotFound, 0 };
 - (void)insertText:(id)string replacementRange:(NSRange)replacementRange
 {
     NSString* characters;
-    NSEvent* event = [NSApp currentEvent];
-    const int mods = translateFlags([event modifierFlags]);
-    const int plain = !(mods & GLFW_MOD_SUPER);
-
     if ([string isKindOfClass:[NSAttributedString class]])
         characters = [string string];
     else
         characters = (NSString*) string;
-
-    NSUInteger i, length = [characters length];
-
-    for (i = 0;  i < length;  i++)
-    {
-        const unichar codepoint = [characters characterAtIndex:i];
-        if ((codepoint & 0xff00) == 0xf700)
-            continue;
-
-        _glfwInputChar(window, codepoint, mods, plain);
-    }
+    snprintf(_glfw.ns.text, sizeof(_glfw.ns.text), "%s", [characters UTF8String]);
+    _glfw.ns.text[sizeof(_glfw.ns.text) - 1] = 0;
 }
 
 - (void)doCommandBySelector:(SEL)selector
@@ -1139,6 +1339,7 @@ int _glfwPlatformCreateWindow(_GLFWwindow* window,
                               const _GLFWctxconfig* ctxconfig,
                               const _GLFWfbconfig* fbconfig)
 {
+    window->ns.deadKeyState = 0;
     if (!initializeAppKit())
         return GLFW_FALSE;
 
@@ -1638,26 +1839,8 @@ void _glfwPlatformSetCursorPos(_GLFWwindow* window, double x, double y)
 
 void _glfwPlatformSetCursorMode(_GLFWwindow* window, int mode)
 {
-    if (mode == GLFW_CURSOR_DISABLED)
-    {
-        _glfw.ns.disabledCursorWindow = window;
-        _glfwPlatformGetCursorPos(window,
-                                  &_glfw.ns.restoreCursorPosX,
-                                  &_glfw.ns.restoreCursorPosY);
-        centerCursor(window);
-        CGAssociateMouseAndMouseCursorPosition(false);
-    }
-    else if (_glfw.ns.disabledCursorWindow == window)
-    {
-        _glfw.ns.disabledCursorWindow = NULL;
-        CGAssociateMouseAndMouseCursorPosition(true);
-        _glfwPlatformSetCursorPos(window,
-                                  _glfw.ns.restoreCursorPosX,
-                                  _glfw.ns.restoreCursorPosY);
-    }
-
-    if (cursorInClientArea(window))
-        updateCursorImage(window);
+    if (_glfwPlatformWindowFocused(window))
+        updateCursorMode(window);
 }
 
 const char* _glfwPlatformGetScancodeName(int scancode)
@@ -1683,16 +1866,7 @@ const char* _glfwPlatformGetScancodeName(int scancode)
     if (!characterCount)
         return NULL;
 
-    CFStringRef string = CFStringCreateWithCharactersNoCopy(kCFAllocatorDefault,
-                                                            characters,
-                                                            characterCount,
-                                                            kCFAllocatorNull);
-    CFStringGetCString(string,
-                       _glfw.ns.keyName,
-                       sizeof(_glfw.ns.keyName),
-                       kCFStringEncodingUTF8);
-    CFRelease(string);
-
+    convert_utf16_to_utf8(characters, characterCount, _glfw.ns.keyName, sizeof(_glfw.ns.keyName));
     return _glfw.ns.keyName;
 }
 
@@ -1906,3 +2080,11 @@ GLFWAPI id glfwGetCocoaWindow(GLFWwindow* handle)
     _GLFW_REQUIRE_INIT_OR_RETURN(nil);
     return window->ns.object;
 }
+
+GLFWAPI GLFWcocoatextinputfilterfun glfwSetCocoaTextInputFilter(GLFWwindow *handle, GLFWcocoatextinputfilterfun callback) {
+    _GLFWwindow* window = (_GLFWwindow*) handle;
+    _GLFW_REQUIRE_INIT_OR_RETURN(nil);
+    GLFWcocoatextinputfilterfun previous = window->ns.textInputFilterCallback;
+    window->ns.textInputFilterCallback = callback;
+    return previous;
+}

glfw/glfw.py

@@ -46,8 +46,11 @@ def init_env(env, pkg_config, at_least_version, module='x11'):
     ans.sources = sinfo['common']['sources'] + sinfo[module]['sources']
     ans.all_headers = [x for x in os.listdir(base) if x.endswith('.h')]
 
+    if module in ('x11', 'wayland'):
+        at_least_version('xkbcommon', 0, 5)
+
     if module == 'x11':
-        for dep in 'x11 xrandr xinerama xcursor xkbcommon xkbcommon-x11'.split():
+        for dep in 'x11 xrandr xinerama xcursor xkbcommon xkbcommon-x11 x11-xcb'.split():
             ans.cflags.extend(pkg_config(dep, '--cflags-only-I'))
             ans.ldpaths.extend(pkg_config(dep, '--libs'))
 
@@ -98,7 +101,7 @@ def collect_source_information():
                 ).group(1).strip().split()
             )
 
-    wayland_protocols = re.search('WaylandProtocols\s+(\S+)\s+', mraw).group(1)
+    wayland_protocols = re.search(r'WaylandProtocols\s+(\S+)\s+', mraw).group(1)
     wayland_protocols = list(map(int, wayland_protocols.split('.')))
     ans = {
         'common': dict(extract_sources('common')),
@@ -193,6 +196,7 @@ def generate_wrappers(glfw_header, glfw_native_header):
     for line in '''\
     void* glfwGetCocoaWindow(GLFWwindow* window)
     uint32_t glfwGetCocoaMonitor(GLFWmonitor* monitor)
+    GLFWcocoatextinputfilterfun glfwSetCocoaTextInputFilter(GLFWwindow* window, GLFWcocoatextinputfilterfun callback)
     void* glfwGetX11Display(void)
     int32_t glfwGetX11Window(GLFWwindow* window)
     void glfwSetX11SelectionString(const char* string)
@@ -209,6 +213,7 @@ def generate_wrappers(glfw_header, glfw_native_header):
 #pragma once
 #include <stddef.h>
 #include <stdint.h>
+typedef int (* GLFWcocoatextinputfilterfun)(int,int,unsigned int);
 {}
 
 {}

glfw/glfw3.h

@@ -1375,9 +1375,15 @@ typedef void (* GLFWcursorenterfun)(GLFWwindow*,int);
  */
 typedef void (* GLFWscrollfun)(GLFWwindow*,double,double);
 
-/*! @brief The function signature for keyboard key callbacks.
+/*! @brief The function signature for key callbacks.
  *
- *  This is the function signature for keyboard key callback functions.
+ *  This is the function signature for key callback functions.
+ *  The semantics of this function are that the key that is interacted with on the
+ *  keyboard is reported, and the text, if any generated by the key is reported.
+ *  So, for example, if on a US-ASCII keyboard the user presses Shift+= GLFW
+ *  will report the text "+" and the key as GLFW_KEY_EQUAL. The reported key takes into
+ *  account any current keyboard maps defined in the OS. So with a dvorak mapping, pressing
+ *  the "s" key will generate text "o" and GLFW_KEY_O.
  *
  *  @param[in] window The window that received the event.
  *  @param[in] key The [keyboard key](@ref keys) that was pressed or released.
@@ -1385,56 +1391,22 @@ typedef void (* GLFWscrollfun)(GLFWwindow*,double,double);
  *  @param[in] action `GLFW_PRESS`, `GLFW_RELEASE` or `GLFW_REPEAT`.
  *  @param[in] mods Bit field describing which [modifier keys](@ref mods) were
  *  held down.
+ *  @param[in] text UTF-8 encoded text generated by this key event or empty string.
+ *  @param[in] reserved Reserved for future use.
+ *
+ *  @note On X11/Wayland if a modifier other than the modifiers GLFW reports
+ *  (ctrl/shift/alt/super) is used, GLFW will report the shifted key rather
+ *  than the unshifted key. So for example, if ISO_Shift_Level_5 is used to
+ *  convert the key A into UP GLFW will report the key as UP with no modifiers.
  *
  *  @sa @ref input_key
- *  @sa @ref glfwSetKeyCallback
+ *  @sa @ref glfwSetKeyboardCallback
  *
- *  @since Added in version 1.0.
- *  @glfw3 Added window handle, scancode and modifier mask parameters.
+ *  @since Added in version 4.0.
  *
  *  @ingroup input
  */
-typedef void (* GLFWkeyfun)(GLFWwindow*,int,int,int,int);
-
-/*! @brief The function signature for Unicode character callbacks.
- *
- *  This is the function signature for Unicode character callback functions.
- *
- *  @param[in] window The window that received the event.
- *  @param[in] codepoint The Unicode code point of the character.
- *
- *  @sa @ref input_char
- *  @sa @ref glfwSetCharCallback
- *
- *  @since Added in version 2.4.
- *  @glfw3 Added window handle parameter.
- *
- *  @ingroup input
- */
-typedef void (* GLFWcharfun)(GLFWwindow*,unsigned int);
-
-/*! @brief The function signature for Unicode character with modifiers
- *  callbacks.
- *
- *  This is the function signature for Unicode character with modifiers callback
- *  functions.  It is called for each input character, regardless of what
- *  modifier keys are held down.
- *
- *  @param[in] window The window that received the event.
- *  @param[in] codepoint The Unicode code point of the character.
- *  @param[in] mods Bit field describing which [modifier keys](@ref mods) were
- *  held down.
- *
- *  @sa @ref input_char
- *  @sa @ref glfwSetCharModsCallback
- *
- *  @deprecated Scheduled for removal in version 4.0.
- *
- *  @since Added in version 3.1.
- *
- *  @ingroup input
- */
-typedef void (* GLFWcharmodsfun)(GLFWwindow*,unsigned int,int);
+typedef void (* GLFWkeyboardfun)(GLFWwindow*, int, int, int, int, const char*, int);
 
 /*! @brief The function signature for file drop callbacks.
  *
@@ -4232,121 +4204,11 @@ GLFWAPI void glfwDestroyCursor(GLFWcursor* cursor);
  */
 GLFWAPI void glfwSetCursor(GLFWwindow* window, GLFWcursor* cursor);
 
-/*! @brief Sets the key callback.
- *
- *  This function sets the key callback of the specified window, which is called
- *  when a key is pressed, repeated or released.
- *
- *  The key functions deal with physical keys, with layout independent
- *  [key tokens](@ref keys) named after their values in the standard US keyboard
- *  layout.  If you want to input text, use the
- *  [character callback](@ref glfwSetCharCallback) instead.
- *
- *  When a window loses input focus, it will generate synthetic key release
- *  events for all pressed keys.  You can tell these events from user-generated
- *  events by the fact that the synthetic ones are generated after the focus
- *  loss event has been processed, i.e. after the
- *  [window focus callback](@ref glfwSetWindowFocusCallback) has been called.
- *
- *  The scancode of a key is specific to that platform or sometimes even to that
- *  machine.  Scancodes are intended to allow users to bind keys that don't have
- *  a GLFW key token.  Such keys have `key` set to `GLFW_KEY_UNKNOWN`, their
- *  state is not saved and so it cannot be queried with @ref glfwGetKey.
- *
- *  Sometimes GLFW needs to generate synthetic key events, in which case the
- *  scancode may be zero.
- *
- *  @param[in] window The window whose callback to set.
- *  @param[in] cbfun The new key callback, or `NULL` to remove the currently
- *  set callback.
- *  @return The previously set callback, or `NULL` if no callback was set or the
- *  library had not been [initialized](@ref intro_init).
- *
- *  @errors Possible errors include @ref GLFW_NOT_INITIALIZED.
- *
- *  @thread_safety This function must only be called from the main thread.
- *
- *  @sa @ref input_key
- *
- *  @since Added in version 1.0.
- *  @glfw3 Added window handle parameter and return value.
+/*! @brief Sets the callback for handling keyboard events.
  *
  *  @ingroup input
  */
-GLFWAPI GLFWkeyfun glfwSetKeyCallback(GLFWwindow* window, GLFWkeyfun cbfun);
-
-/*! @brief Sets the Unicode character callback.
- *
- *  This function sets the character callback of the specified window, which is
- *  called when a Unicode character is input.
- *
- *  The character callback is intended for Unicode text input.  As it deals with
- *  characters, it is keyboard layout dependent, whereas the
- *  [key callback](@ref glfwSetKeyCallback) is not.  Characters do not map 1:1
- *  to physical keys, as a key may produce zero, one or more characters.  If you
- *  want to know whether a specific physical key was pressed or released, see
- *  the key callback instead.
- *
- *  The character callback behaves as system text input normally does and will
- *  not be called if modifier keys are held down that would prevent normal text
- *  input on that platform, for example a Super (Command) key on macOS or Alt key
- *  on Windows.  There is a
- *  [character with modifiers callback](@ref glfwSetCharModsCallback) that
- *  receives these events.
- *
- *  @param[in] window The window whose callback to set.
- *  @param[in] cbfun The new callback, or `NULL` to remove the currently set
- *  callback.
- *  @return The previously set callback, or `NULL` if no callback was set or the
- *  library had not been [initialized](@ref intro_init).
- *
- *  @errors Possible errors include @ref GLFW_NOT_INITIALIZED.
- *
- *  @thread_safety This function must only be called from the main thread.
- *
- *  @sa @ref input_char
- *
- *  @since Added in version 2.4.
- *  @glfw3 Added window handle parameter and return value.
- *
- *  @ingroup input
- */
-GLFWAPI GLFWcharfun glfwSetCharCallback(GLFWwindow* window, GLFWcharfun cbfun);
-
-/*! @brief Sets the Unicode character with modifiers callback.
- *
- *  This function sets the character with modifiers callback of the specified
- *  window, which is called when a Unicode character is input regardless of what
- *  modifier keys are used.
- *
- *  The character with modifiers callback is intended for implementing custom
- *  Unicode character input.  For regular Unicode text input, see the
- *  [character callback](@ref glfwSetCharCallback).  Like the character
- *  callback, the character with modifiers callback deals with characters and is
- *  keyboard layout dependent.  Characters do not map 1:1 to physical keys, as
- *  a key may produce zero, one or more characters.  If you want to know whether
- *  a specific physical key was pressed or released, see the
- *  [key callback](@ref glfwSetKeyCallback) instead.
- *
- *  @param[in] window The window whose callback to set.
- *  @param[in] cbfun The new callback, or `NULL` to remove the currently set
- *  callback.
- *  @return The previously set callback, or `NULL` if no callback was set or an
- *  [error](@ref error_handling) occurred.
- *
- *  @deprecated Scheduled for removal in version 4.0.
- *
- *  @errors Possible errors include @ref GLFW_NOT_INITIALIZED.
- *
- *  @thread_safety This function must only be called from the main thread.
- *
- *  @sa @ref input_char
- *
- *  @since Added in version 3.1.
- *
- *  @ingroup input
- */
-GLFWAPI GLFWcharmodsfun glfwSetCharModsCallback(GLFWwindow* window, GLFWcharmodsfun cbfun);
+GLFWAPI GLFWkeyboardfun glfwSetKeyboardCallback(GLFWwindow* window, GLFWkeyboardfun cbfun);
 
 /*! @brief Sets the mouse button callback.
  *
@@ -4720,7 +4582,7 @@ GLFWAPI const char* glfwGetJoystickGUID(int jid);
  *  This function may be called from the joystick callback, even for a joystick
  *  that is being disconnected.
  *
- *  @param[in] joystick The joystick whose pointer to set.
+ *  @param[in] jid The joystick whose pointer to set.
  *  @param[in] pointer The new value.
  *
  *  @errors Possible errors include @ref GLFW_NOT_INITIALIZED.
@@ -4745,7 +4607,7 @@ GLFWAPI void glfwSetJoystickUserPointer(int jid, void* pointer);
  *  This function may be called from the joystick callback, even for a joystick
  *  that is being disconnected.
  *
- *  @param[in] joystick The joystick whose pointer to return.
+ *  @param[in] jid The joystick whose pointer to return.
  *
  *  @errors Possible errors include @ref GLFW_NOT_INITIALIZED.
  *

glfw/init.c

@@ -57,37 +57,6 @@ static _GLFWinitconfig _glfwInitHints =
     }
 };
 
-// Returns a generic string representation of the specified error
-//
-static const char* getErrorString(int code)
-{
-    switch (code)
-    {
-        case GLFW_NOT_INITIALIZED:
-            return "The GLFW library is not initialized";
-        case GLFW_NO_CURRENT_CONTEXT:
-            return "There is no current context";
-        case GLFW_INVALID_ENUM:
-            return "Invalid argument for enum parameter";
-        case GLFW_INVALID_VALUE:
-            return "Invalid value for parameter";
-        case GLFW_OUT_OF_MEMORY:
-            return "Out of memory";
-        case GLFW_API_UNAVAILABLE:
-            return "The requested API is unavailable";
-        case GLFW_VERSION_UNAVAILABLE:
-            return "The requested API version is unavailable";
-        case GLFW_PLATFORM_ERROR:
-            return "An undocumented platform-specific error occurred";
-        case GLFW_FORMAT_UNAVAILABLE:
-            return "The requested format is unavailable";
-        case GLFW_NO_WINDOW_CONTEXT:
-            return "The specified window has no context";
-        default:
-            return "ERROR: UNKNOWN GLFW ERROR";
-    }
-}
-
 // Terminate the library
 //
 static void terminate(void)
@@ -173,7 +142,30 @@ void _glfwInputError(int code, const char* format, ...)
         description[sizeof(description) - 1] = '\0';
     }
     else
-        strcpy(description, getErrorString(code));
+    {
+        if (code == GLFW_NOT_INITIALIZED)
+            strcpy(description, "The GLFW library is not initialized");
+        else if (code == GLFW_NO_CURRENT_CONTEXT)
+            strcpy(description, "There is no current context");
+        else if (code == GLFW_INVALID_ENUM)
+            strcpy(description, "Invalid argument for enum parameter");
+        else if (code == GLFW_INVALID_VALUE)
+            strcpy(description, "Invalid value for parameter");
+        else if (code == GLFW_OUT_OF_MEMORY)
+            strcpy(description, "Out of memory");
+        else if (code == GLFW_API_UNAVAILABLE)
+            strcpy(description, "The requested API is unavailable");
+        else if (code == GLFW_VERSION_UNAVAILABLE)
+            strcpy(description, "The requested API version is unavailable");
+        else if (code == GLFW_PLATFORM_ERROR)
+            strcpy(description, "A platform-specific error occurred");
+        else if (code == GLFW_FORMAT_UNAVAILABLE)
+            strcpy(description, "The requested format is unavailable");
+        else if (code == GLFW_NO_WINDOW_CONTEXT)
+            strcpy(description, "The specified window has no context");
+        else
+            strcpy(description, "ERROR: UNKNOWN GLFW ERROR");
+    }
 
     if (_glfw.initialized)
     {

glfw/input.c

@@ -32,7 +32,6 @@
 #include <math.h>
 #include <stdlib.h>
 #include <string.h>
-#include <ctype.h>
 
 // Internal key state used for sticky keys
 #define _GLFW_STICK 3
@@ -255,9 +254,9 @@ static GLFWbool parseMapping(_GLFWmapping* mapping, const char* string)
 //////                         GLFW event API                       //////
 //////////////////////////////////////////////////////////////////////////
 
-// Notifies shared code of a physical key event
+// Notifies shared code of a key event
 //
-void _glfwInputKey(_GLFWwindow* window, int key, int scancode, int action, int mods)
+void _glfwInputKeyboard(_GLFWwindow* window, int key, int scancode, int action, int mods, const char* text, int state)
 {
     if (key >= 0 && key <= GLFW_KEY_LAST)
     {
@@ -278,31 +277,10 @@ void _glfwInputKey(_GLFWwindow* window, int key, int scancode, int action, int m
             action = GLFW_REPEAT;
     }
 
-    if (!window->lockKeyMods)
-        mods &= ~(GLFW_MOD_CAPS_LOCK | GLFW_MOD_NUM_LOCK);
 
-    if (window->callbacks.key)
-        window->callbacks.key((GLFWwindow*) window, key, scancode, action, mods);
-}
-
-// Notifies shared code of a Unicode codepoint input event
-// The 'plain' parameter determines whether to emit a regular character event
-//
-void _glfwInputChar(_GLFWwindow* window, unsigned int codepoint, int mods, GLFWbool plain)
-{
-    if (codepoint < 32 || (codepoint > 126 && codepoint < 160))
-        return;
-
-    if (!window->lockKeyMods)
-        mods &= ~(GLFW_MOD_CAPS_LOCK | GLFW_MOD_NUM_LOCK);
-
-    if (window->callbacks.charmods)
-        window->callbacks.charmods((GLFWwindow*) window, codepoint, mods);
-
-    if (plain)
-    {
-        if (window->callbacks.character)
-            window->callbacks.character((GLFWwindow*) window, codepoint);
+    if (window->callbacks.keyboard) {
+        if (!window->lockKeyMods) mods &= ~(GLFW_MOD_CAPS_LOCK | GLFW_MOD_NUM_LOCK);
+        window->callbacks.keyboard((GLFWwindow*) window, key, scancode, action, mods, text, state);
     }
 }
 
@@ -454,6 +432,137 @@ void _glfwFreeJoystick(_GLFWjoystick* js)
     memset(js, 0, sizeof(_GLFWjoystick));
 }
 
+const char* _glfwGetKeyName(int key)
+{
+    switch (key)
+    {
+        // Printable keys
+        case GLFW_KEY_A:            return "A";
+        case GLFW_KEY_B:            return "B";
+        case GLFW_KEY_C:            return "C";
+        case GLFW_KEY_D:            return "D";
+        case GLFW_KEY_E:            return "E";
+        case GLFW_KEY_F:            return "F";
+        case GLFW_KEY_G:            return "G";
+        case GLFW_KEY_H:            return "H";
+        case GLFW_KEY_I:            return "I";
+        case GLFW_KEY_J:            return "J";
+        case GLFW_KEY_K:            return "K";
+        case GLFW_KEY_L:            return "L";
+        case GLFW_KEY_M:            return "M";
+        case GLFW_KEY_N:            return "N";
+        case GLFW_KEY_O:            return "O";
+        case GLFW_KEY_P:            return "P";
+        case GLFW_KEY_Q:            return "Q";
+        case GLFW_KEY_R:            return "R";
+        case GLFW_KEY_S:            return "S";
+        case GLFW_KEY_T:            return "T";
+        case GLFW_KEY_U:            return "U";
+        case GLFW_KEY_V:            return "V";
+        case GLFW_KEY_W:            return "W";
+        case GLFW_KEY_X:            return "X";
+        case GLFW_KEY_Y:            return "Y";
+        case GLFW_KEY_Z:            return "Z";
+        case GLFW_KEY_1:            return "1";
+        case GLFW_KEY_2:            return "2";
+        case GLFW_KEY_3:            return "3";
+        case GLFW_KEY_4:            return "4";
+        case GLFW_KEY_5:            return "5";
+        case GLFW_KEY_6:            return "6";
+        case GLFW_KEY_7:            return "7";
+        case GLFW_KEY_8:            return "8";
+        case GLFW_KEY_9:            return "9";
+        case GLFW_KEY_0:            return "0";
+        case GLFW_KEY_SPACE:        return "SPACE";
+        case GLFW_KEY_MINUS:        return "MINUS";
+        case GLFW_KEY_EQUAL:        return "EQUAL";
+        case GLFW_KEY_LEFT_BRACKET: return "LEFT BRACKET";
+        case GLFW_KEY_RIGHT_BRACKET: return "RIGHT BRACKET";
+        case GLFW_KEY_BACKSLASH:    return "BACKSLASH";
+        case GLFW_KEY_SEMICOLON:    return "SEMICOLON";
+        case GLFW_KEY_APOSTROPHE:   return "APOSTROPHE";
+        case GLFW_KEY_GRAVE_ACCENT: return "GRAVE ACCENT";
+        case GLFW_KEY_COMMA:        return "COMMA";
+        case GLFW_KEY_PERIOD:       return "PERIOD";
+        case GLFW_KEY_SLASH:        return "SLASH";
+        case GLFW_KEY_WORLD_1:      return "WORLD 1";
+        case GLFW_KEY_WORLD_2:      return "WORLD 2";
+
+        // Function keys
+        case GLFW_KEY_ESCAPE:       return "ESCAPE";
+        case GLFW_KEY_F1:           return "F1";
+        case GLFW_KEY_F2:           return "F2";
+        case GLFW_KEY_F3:           return "F3";
+        case GLFW_KEY_F4:           return "F4";
+        case GLFW_KEY_F5:           return "F5";
+        case GLFW_KEY_F6:           return "F6";
+        case GLFW_KEY_F7:           return "F7";
+        case GLFW_KEY_F8:           return "F8";
+        case GLFW_KEY_F9:           return "F9";
+        case GLFW_KEY_F10:          return "F10";
+        case GLFW_KEY_F11:          return "F11";
+        case GLFW_KEY_F12:          return "F12";
+        case GLFW_KEY_F13:          return "F13";
+        case GLFW_KEY_F14:          return "F14";
+        case GLFW_KEY_F15:          return "F15";
+        case GLFW_KEY_F16:          return "F16";
+        case GLFW_KEY_F17:          return "F17";
+        case GLFW_KEY_F18:          return "F18";
+        case GLFW_KEY_F19:          return "F19";
+        case GLFW_KEY_F20:          return "F20";
+        case GLFW_KEY_F21:          return "F21";
+        case GLFW_KEY_F22:          return "F22";
+        case GLFW_KEY_F23:          return "F23";
+        case GLFW_KEY_F24:          return "F24";
+        case GLFW_KEY_F25:          return "F25";
+        case GLFW_KEY_UP:           return "UP";
+        case GLFW_KEY_DOWN:         return "DOWN";
+        case GLFW_KEY_LEFT:         return "LEFT";
+        case GLFW_KEY_RIGHT:        return "RIGHT";
+        case GLFW_KEY_LEFT_SHIFT:   return "LEFT SHIFT";
+        case GLFW_KEY_RIGHT_SHIFT:  return "RIGHT SHIFT";
+        case GLFW_KEY_LEFT_CONTROL: return "LEFT CONTROL";
+        case GLFW_KEY_RIGHT_CONTROL: return "RIGHT CONTROL";
+        case GLFW_KEY_LEFT_ALT:     return "LEFT ALT";
+        case GLFW_KEY_RIGHT_ALT:    return "RIGHT ALT";
+        case GLFW_KEY_TAB:          return "TAB";
+        case GLFW_KEY_ENTER:        return "ENTER";
+        case GLFW_KEY_BACKSPACE:    return "BACKSPACE";
+        case GLFW_KEY_INSERT:       return "INSERT";
+        case GLFW_KEY_DELETE:       return "DELETE";
+        case GLFW_KEY_PAGE_UP:      return "PAGE UP";
+        case GLFW_KEY_PAGE_DOWN:    return "PAGE DOWN";
+        case GLFW_KEY_HOME:         return "HOME";
+        case GLFW_KEY_END:          return "END";
+        case GLFW_KEY_KP_0:         return "KEYPAD 0";
+        case GLFW_KEY_KP_1:         return "KEYPAD 1";
+        case GLFW_KEY_KP_2:         return "KEYPAD 2";
+        case GLFW_KEY_KP_3:         return "KEYPAD 3";
+        case GLFW_KEY_KP_4:         return "KEYPAD 4";
+        case GLFW_KEY_KP_5:         return "KEYPAD 5";
+        case GLFW_KEY_KP_6:         return "KEYPAD 6";
+        case GLFW_KEY_KP_7:         return "KEYPAD 7";
+        case GLFW_KEY_KP_8:         return "KEYPAD 8";
+        case GLFW_KEY_KP_9:         return "KEYPAD 9";
+        case GLFW_KEY_KP_DIVIDE:    return "KEYPAD DIVIDE";
+        case GLFW_KEY_KP_MULTIPLY:  return "KEYPAD MULTPLY";
+        case GLFW_KEY_KP_SUBTRACT:  return "KEYPAD SUBTRACT";
+        case GLFW_KEY_KP_ADD:       return "KEYPAD ADD";
+        case GLFW_KEY_KP_DECIMAL:   return "KEYPAD DECIMAL";
+        case GLFW_KEY_KP_EQUAL:     return "KEYPAD EQUAL";
+        case GLFW_KEY_KP_ENTER:     return "KEYPAD ENTER";
+        case GLFW_KEY_PRINT_SCREEN: return "PRINT SCREEN";
+        case GLFW_KEY_NUM_LOCK:     return "NUM LOCK";
+        case GLFW_KEY_CAPS_LOCK:    return "CAPS LOCK";
+        case GLFW_KEY_SCROLL_LOCK:  return "SCROLL LOCK";
+        case GLFW_KEY_PAUSE:        return "PAUSE";
+        case GLFW_KEY_LEFT_SUPER:   return "LEFT SUPER";
+        case GLFW_KEY_RIGHT_SUPER:  return "RIGHT SUPER";
+        case GLFW_KEY_MENU:         return "MENU";
+
+        default:                    return "UNKNOWN";
+    }
+}
 
 //////////////////////////////////////////////////////////////////////////
 //////                        GLFW public API                       //////
@@ -509,9 +618,7 @@ GLFWAPI void glfwSetInputMode(GLFWwindow* handle, int mode, int value)
         _glfwPlatformGetCursorPos(window,
                                   &window->virtualCursorPosX,
                                   &window->virtualCursorPosY);
-
-        if (_glfwPlatformWindowFocused(window))
-            _glfwPlatformSetCursorMode(window, value);
+        _glfwPlatformSetCursorMode(window, value);
     }
     else if (mode == GLFW_STICKY_KEYS)
     {
@@ -791,33 +898,13 @@ GLFWAPI void glfwSetCursor(GLFWwindow* windowHandle, GLFWcursor* cursorHandle)
     _glfwPlatformSetCursor(window, cursor);
 }
 
-GLFWAPI GLFWkeyfun glfwSetKeyCallback(GLFWwindow* handle, GLFWkeyfun cbfun)
+GLFWAPI GLFWkeyboardfun glfwSetKeyboardCallback(GLFWwindow* handle, GLFWkeyboardfun cbfun)
 {
     _GLFWwindow* window = (_GLFWwindow*) handle;
     assert(window != NULL);
 
     _GLFW_REQUIRE_INIT_OR_RETURN(NULL);
-    _GLFW_SWAP_POINTERS(window->callbacks.key, cbfun);
-    return cbfun;
-}
-
-GLFWAPI GLFWcharfun glfwSetCharCallback(GLFWwindow* handle, GLFWcharfun cbfun)
-{
-    _GLFWwindow* window = (_GLFWwindow*) handle;
-    assert(window != NULL);
-
-    _GLFW_REQUIRE_INIT_OR_RETURN(NULL);
-    _GLFW_SWAP_POINTERS(window->callbacks.character, cbfun);
-    return cbfun;
-}
-
-GLFWAPI GLFWcharmodsfun glfwSetCharModsCallback(GLFWwindow* handle, GLFWcharmodsfun cbfun)
-{
-    _GLFWwindow* window = (_GLFWwindow*) handle;
-    assert(window != NULL);
-
-    _GLFW_REQUIRE_INIT_OR_RETURN(NULL);
-    _GLFW_SWAP_POINTERS(window->callbacks.charmods, cbfun);
+    _GLFW_SWAP_POINTERS(window->callbacks.keyboard, cbfun);
     return cbfun;
 }
 
@@ -1088,7 +1175,9 @@ GLFWAPI int glfwUpdateGamepadMappings(const char* string)
 
     while (*c)
     {
-        if (isxdigit(*c))
+        if ((*c >= '0' && *c <= '9') ||
+            (*c >= 'a' && *c <= 'f') ||
+            (*c >= 'A' && *c <= 'F'))
         {
             char line[1024];
 
@@ -1305,4 +1394,3 @@ GLFWAPI uint64_t glfwGetTimerFrequency(void)
     _GLFW_REQUIRE_INIT_OR_RETURN(0);
     return _glfwPlatformGetTimerFrequency();
 }
-

glfw/internal.h

@@ -407,9 +407,7 @@ struct _GLFWwindow
         GLFWcursorposfun        cursorPos;
         GLFWcursorenterfun      cursorEnter;
         GLFWscrollfun           scroll;
-        GLFWkeyfun              key;
-        GLFWcharfun             character;
-        GLFWcharmodsfun         charmods;
+        GLFWkeyboardfun         keyboard;
         GLFWdropfun             drop;
     } callbacks;
 
@@ -711,10 +709,7 @@ void _glfwInputWindowDamage(_GLFWwindow* window);
 void _glfwInputWindowCloseRequest(_GLFWwindow* window);
 void _glfwInputWindowMonitor(_GLFWwindow* window, _GLFWmonitor* monitor);
 
-void _glfwInputKey(_GLFWwindow* window,
-                   int key, int scancode, int action, int mods);
-void _glfwInputChar(_GLFWwindow* window,
-                    unsigned int codepoint, int mods, GLFWbool plain);
+void _glfwInputKeyboard(_GLFWwindow* window, int key, int scancode, int action, int mods, const char* text, int state);
 void _glfwInputScroll(_GLFWwindow* window, double xoffset, double yoffset);
 void _glfwInputMouseClick(_GLFWwindow* window, int button, int action, int mods);
 void _glfwInputCursorPos(_GLFWwindow* window, double xpos, double ypos);
@@ -763,6 +758,7 @@ _GLFWjoystick* _glfwAllocJoystick(const char* name,
                                   int buttonCount,
                                   int hatCount);
 void _glfwFreeJoystick(_GLFWjoystick* js);
+const char* _glfwGetKeyName(int key);
 
 GLFWbool _glfwInitVulkan(int mode);
 void _glfwTerminateVulkan(void);

glfw/source-info.json

@@ -58,7 +58,6 @@
       "posix_time.h", 
       "posix_thread.h", 
       "xkb_glfw.h", 
-      "xkb_unicode.h", 
       "egl_context.h", 
       "osmesa_context.h", 
       "linux_joystick.h"
@@ -76,7 +75,7 @@
       "wl_window.c", 
       "posix_time.c", 
       "posix_thread.c", 
-      "xkb_unicode.c", 
+      "xkb_glfw.c", 
       "egl_context.c", 
       "osmesa_context.c", 
       "linux_joystick.c"
@@ -109,7 +108,7 @@
   "x11": {
     "headers": [
       "x11_platform.h", 
-      "xkb_unicode.h", 
+      "xkb_glfw.h", 
       "posix_time.h", 
       "posix_thread.h", 
       "glx_context.h", 
@@ -121,7 +120,7 @@
       "x11_init.c", 
       "x11_monitor.c", 
       "x11_window.c", 
-      "xkb_unicode.c", 
+      "xkb_glfw.c", 
       "posix_time.c", 
       "posix_thread.c", 
       "glx_context.c", 

glfw/win32_platform.h

@@ -120,6 +120,8 @@ typedef struct
     HRGN hRgnBlur;
     BOOL fTransitionOnMaximized;
 } DWM_BLURBEHIND;
+#else
+ #include <dwmapi.h>
 #endif /*Windows Vista*/
 
 #ifndef DPI_ENUMS_DECLARED
@@ -240,7 +242,9 @@ typedef VkBool32 (APIENTRY *PFN_vkGetPhysicalDeviceWin32PresentationSupportKHR)(
 #include "egl_context.h"
 #include "osmesa_context.h"
 
-#define _GLFW_WNDCLASSNAME L"GLFW30"
+#if !defined(_GLFW_WNDCLASSNAME)
+ #define _GLFW_WNDCLASSNAME L"GLFW30"
+#endif
 
 #define _glfw_dlopen(name) LoadLibraryA(name)
 #define _glfw_dlclose(handle) FreeLibrary((HMODULE) handle)

glfw/win32_window.c

@@ -235,26 +235,6 @@ static void centerCursor(_GLFWwindow* window)
     _glfwPlatformSetCursorPos(window, width / 2.0, height / 2.0);
 }
 
-// Returns whether the cursor is in the client area of the specified window
-//
-static GLFWbool cursorInClientArea(_GLFWwindow* window)
-{
-    RECT area;
-    POINT pos;
-
-    if (!GetCursorPos(&pos))
-        return GLFW_FALSE;
-
-    if (WindowFromPoint(pos) != window->win32.handle)
-        return GLFW_FALSE;
-
-    GetClientRect(window->win32.handle, &area);
-    ClientToScreen(window->win32.handle, (POINT*) &area.left);
-    ClientToScreen(window->win32.handle, (POINT*) &area.right);
-
-    return PtInRect(&area, pos);
-}
-
 // Updates the cursor image according to its cursor mode
 //
 static void updateCursorImage(_GLFWwindow* window)
@@ -286,6 +266,67 @@ static void updateClipRect(_GLFWwindow* window)
         ClipCursor(NULL);
 }
 
+// Apply disabled cursor mode to a focused window
+//
+static void disableCursor(_GLFWwindow* window)
+{
+    const RAWINPUTDEVICE rid = { 0x01, 0x02, 0, window->win32.handle };
+
+    _glfw.win32.disabledCursorWindow = window;
+    _glfwPlatformGetCursorPos(window,
+                              &_glfw.win32.restoreCursorPosX,
+                              &_glfw.win32.restoreCursorPosY);
+    updateCursorImage(window);
+    centerCursor(window);
+    updateClipRect(window);
+
+    if (!RegisterRawInputDevices(&rid, 1, sizeof(rid)))
+    {
+        _glfwInputErrorWin32(GLFW_PLATFORM_ERROR,
+                             "Win32: Failed to register raw input device");
+    }
+}
+
+// Exit disabled cursor mode for the specified window
+//
+static void enableCursor(_GLFWwindow* window)
+{
+    const RAWINPUTDEVICE rid = { 0x01, 0x02, RIDEV_REMOVE, NULL };
+
+    _glfw.win32.disabledCursorWindow = NULL;
+    updateClipRect(NULL);
+    _glfwPlatformSetCursorPos(window,
+                              _glfw.win32.restoreCursorPosX,
+                              _glfw.win32.restoreCursorPosY);
+    updateCursorImage(window);
+
+    if (!RegisterRawInputDevices(&rid, 1, sizeof(rid)))
+    {
+        _glfwInputErrorWin32(GLFW_PLATFORM_ERROR,
+                             "Win32: Failed to remove raw input device");
+    }
+}
+
+// Returns whether the cursor is in the client area of the specified window
+//
+static GLFWbool cursorInClientArea(_GLFWwindow* window)
+{
+    RECT area;
+    POINT pos;
+
+    if (!GetCursorPos(&pos))
+        return GLFW_FALSE;
+
+    if (WindowFromPoint(pos) != window->win32.handle)
+        return GLFW_FALSE;
+
+    GetClientRect(window->win32.handle, &area);
+    ClientToScreen(window->win32.handle, (POINT*) &area.left);
+    ClientToScreen(window->win32.handle, (POINT*) &area.right);
+
+    return PtInRect(&area, pos);
+}
+
 // Update native window styles to match attributes
 //
 static void updateWindowStyles(const _GLFWwindow* window)
@@ -575,7 +616,7 @@ static LRESULT CALLBACK windowProc(HWND hWnd, UINT uMsg,
             if (lParam == 0 && window->win32.frameAction)
             {
                 if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                    _glfwPlatformSetCursorMode(window, GLFW_CURSOR_DISABLED);
+                    disableCursor(window);
 
                 window->win32.frameAction = GLFW_FALSE;
             }
@@ -593,7 +634,7 @@ static LRESULT CALLBACK windowProc(HWND hWnd, UINT uMsg,
                 break;
 
             if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_DISABLED);
+                disableCursor(window);
 
             return 0;
         }
@@ -601,7 +642,7 @@ static LRESULT CALLBACK windowProc(HWND hWnd, UINT uMsg,
         case WM_KILLFOCUS:
         {
             if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_NORMAL);
+                enableCursor(window);
 
             if (window->monitor && window->autoIconify)
                 _glfwPlatformIconifyWindow(window);
@@ -857,10 +898,10 @@ static LRESULT CALLBACK windowProc(HWND hWnd, UINT uMsg,
         case WM_ENTERSIZEMOVE:
         case WM_ENTERMENULOOP:
         {
-            // HACK: Postpone cursor disabling while the user is moving or
-            //       resizing the window or using the menu
+            // HACK: Enable the cursor while the user is moving or
+            //       resizing the window or using the window menu
             if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_NORMAL);
+                enableCursor(window);
 
             break;
         }
@@ -871,7 +912,7 @@ static LRESULT CALLBACK windowProc(HWND hWnd, UINT uMsg,
             // HACK: Disable the cursor once the user is done moving or
             //       resizing the window or using the menu
             if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_DISABLED);
+                disableCursor(window);
 
             break;
         }
@@ -1777,39 +1818,12 @@ void _glfwPlatformSetCursorMode(_GLFWwindow* window, int mode)
 {
     if (mode == GLFW_CURSOR_DISABLED)
     {
-        const RAWINPUTDEVICE rid = { 0x01, 0x02, 0, window->win32.handle };
-
-        _glfw.win32.disabledCursorWindow = window;
-        _glfwPlatformGetCursorPos(window,
-                                  &_glfw.win32.restoreCursorPosX,
-                                  &_glfw.win32.restoreCursorPosY);
-        centerCursor(window);
-        updateClipRect(window);
-
-        if (!RegisterRawInputDevices(&rid, 1, sizeof(rid)))
-        {
-            _glfwInputErrorWin32(GLFW_PLATFORM_ERROR,
-                                 "Win32: Failed to register raw input device");
-        }
+        if (_glfwPlatformWindowFocused(window))
+            disableCursor(window);
     }
     else if (_glfw.win32.disabledCursorWindow == window)
-    {
-        const RAWINPUTDEVICE rid = { 0x01, 0x02, RIDEV_REMOVE, NULL };
-
-        _glfw.win32.disabledCursorWindow = NULL;
-        updateClipRect(NULL);
-        _glfwPlatformSetCursorPos(window,
-                                  _glfw.win32.restoreCursorPosX,
-                                  _glfw.win32.restoreCursorPosY);
-
-        if (!RegisterRawInputDevices(&rid, 1, sizeof(rid)))
-        {
-            _glfwInputErrorWin32(GLFW_PLATFORM_ERROR,
-                                 "Win32: Failed to remove raw input device");
-        }
-    }
-
-    if (cursorInClientArea(window))
+        enableCursor(window);
+    else if (cursorInClientArea(window))
         updateCursorImage(window);
 }
 

glfw/window.c

@@ -54,7 +54,7 @@ void _glfwInputWindowFocus(_GLFWwindow* window, GLFWbool focused)
             if (window->keys[key] == GLFW_PRESS)
             {
                 const int scancode = _glfwPlatformGetKeyScancode(key);
-                _glfwInputKey(window, key, scancode, GLFW_RELEASE, 0);
+                _glfwInputKeyboard(window, key, scancode, GLFW_RELEASE, 0, "", 0);
             }
         }
 

glfw/wl_init.c

@@ -362,7 +362,7 @@ static void keyboardHandleKeymap(void* data,
         close(fd);
         return;
     }
-    xkb_glfw_compile_keymap(mapStr);
+    glfw_xkb_compile_keymap(&_glfw.wl.xkb, mapStr);
     munmap(mapStr, size);
     close(fd);
 
@@ -404,54 +404,6 @@ static void keyboardHandleLeave(void* data,
     _glfwInputWindowFocus(window, GLFW_FALSE);
 }
 
-static xkb_keysym_t composeSymbol(xkb_keysym_t sym)
-{
-    if (sym == XKB_KEY_NoSymbol || !_glfw.wl.xkb.composeState)
-        return sym;
-    if (xkb_compose_state_feed(_glfw.wl.xkb.composeState, sym)
-            != XKB_COMPOSE_FEED_ACCEPTED)
-        return sym;
-    switch (xkb_compose_state_get_status(_glfw.wl.xkb.composeState))
-    {
-        case XKB_COMPOSE_COMPOSED:
-            return xkb_compose_state_get_one_sym(_glfw.wl.xkb.composeState);
-        case XKB_COMPOSE_COMPOSING:
-        case XKB_COMPOSE_CANCELLED:
-            return XKB_KEY_NoSymbol;
-        case XKB_COMPOSE_NOTHING:
-        default:
-            return sym;
-    }
-}
-
-static void inputChar(_GLFWwindow* window, uint32_t key, GLFWbool *shouldRepeat)
-{
-    uint32_t code, numSyms;
-    long cp;
-    const xkb_keysym_t *syms;
-    xkb_keysym_t sym;
-
-    code = key + 8;
-    numSyms = xkb_state_key_get_syms(_glfw.wl.xkb.state, code, &syms);
-    *shouldRepeat = xkb_keymap_key_repeats(_glfw.wl.xkb.keymap, code);
-
-    if (numSyms == 1)
-    {
-        sym = composeSymbol(syms[0]);
-        cp = _glfwKeySym2Unicode(sym);
-        if (cp != -1)
-        {
-            const int mods = _glfw.wl.xkb.modifiers;
-            const int plain = !(mods & (GLFW_MOD_CONTROL | GLFW_MOD_ALT));
-            if (*shouldRepeat) {
-                _glfw.wl.keyRepeatInfo.codepoint = cp;
-                _glfw.wl.keyRepeatInfo.plain = plain;
-            }
-            _glfwInputChar(window, cp, mods, plain);
-        }
-    }
-}
-
 static void keyboardHandleKey(void* data,
                               struct wl_keyboard* keyboard,
                               uint32_t serial,
@@ -459,34 +411,18 @@ static void keyboardHandleKey(void* data,
                               uint32_t key,
                               uint32_t state)
 {
-    int keyCode;
-    int action;
     _GLFWwindow* window = _glfw.wl.keyboardFocus;
-
     if (!window)
         return;
-
-    keyCode = xkb_glfw_to_glfw_key_code(key);
-    action = state == WL_KEYBOARD_KEY_STATE_PRESSED
-            ? GLFW_PRESS : GLFW_RELEASE;
+    int action = state == WL_KEYBOARD_KEY_STATE_PRESSED ? GLFW_PRESS : GLFW_RELEASE;
+    glfw_xkb_handle_key_event(window, &_glfw.wl.xkb, key, action);
     _glfw.wl.keyRepeatInfo.nextRepeatAt = 0;
-    _glfw.wl.keyRepeatInfo.codepoint = -1;
 
-    _glfwInputKey(window, keyCode, key, action,
-                  _glfw.wl.xkb.modifiers);
-
-    if (action == GLFW_PRESS)
+    if (action == GLFW_PRESS && _glfw.wl.keyboardRepeatRate > 0 && glfw_xkb_should_repeat(&_glfw.wl.xkb, key))
     {
-        GLFWbool shouldRepeat = GLFW_FALSE;
-        inputChar(window, key, &shouldRepeat);
-
-        if (shouldRepeat && _glfw.wl.keyboardRepeatRate > 0)
-        {
-            _glfw.wl.keyRepeatInfo.glfwKeyCode = keyCode;
-            _glfw.wl.keyRepeatInfo.scancode = key;
-            _glfw.wl.keyRepeatInfo.nextRepeatAt = glfwGetTime() + (double)(_glfw.wl.keyboardRepeatDelay) / 1000.0;
-            _glfw.wl.keyRepeatInfo.keyboardFocus = window;
-        }
+        _glfw.wl.keyRepeatInfo.key = key;
+        _glfw.wl.keyRepeatInfo.nextRepeatAt = glfwGetTime() + (double)(_glfw.wl.keyboardRepeatDelay) / 1000.0;
+        _glfw.wl.keyRepeatInfo.keyboardFocus = window;
     }
 }
 
@@ -498,7 +434,7 @@ static void keyboardHandleModifiers(void* data,
                                     uint32_t modsLocked,
                                     uint32_t group)
 {
-    xkb_glfw_update_modifiers(modsDepressed, modsLatched, modsLocked, group);
+    glfw_xkb_update_modifiers(&_glfw.wl.xkb, modsDepressed, modsLatched, modsLocked, 0, 0, group);
 }
 
 static void keyboardHandleRepeatInfo(void* data,
@@ -711,8 +647,6 @@ int _glfwPlatformInit(void)
     _glfw.wl.egl.window_resize = (PFN_wl_egl_window_resize)
         _glfw_dlsym(_glfw.wl.egl.handle, "wl_egl_window_resize");
 
-    load_glfw_xkb();
-
     _glfw.wl.display = wl_display_connect(NULL);
     if (!_glfw.wl.display)
     {
@@ -724,7 +658,7 @@ int _glfwPlatformInit(void)
     _glfw.wl.registry = wl_display_get_registry(_glfw.wl.display);
     wl_registry_add_listener(_glfw.wl.registry, &registryListener, NULL);
 
-    create_glfw_xkb_context();
+    if (!glfw_xkb_create_context(&_glfw.wl.xkb)) return GLFW_FALSE;
 
     // Sync so we got all registry objects
     wl_display_roundtrip(_glfw.wl.display);
@@ -767,7 +701,7 @@ void _glfwPlatformTerminate(void)
         _glfw.wl.egl.handle = NULL;
     }
 
-    release_glfw_xkb();
+    glfw_xkb_release(&_glfw.wl.xkb);
 
     if (_glfw.wl.cursorTheme)
         wl_cursor_theme_destroy(_glfw.wl.cursorTheme);

glfw/wl_platform.h

@@ -48,9 +48,7 @@ typedef VkBool32 (APIENTRY *PFN_vkGetPhysicalDeviceWaylandPresentationSupportKHR
 #else
 #include "null_joystick.h"
 #endif
-#define GLFW_XKB_GLOBAL_NAME _glfw.wl.xkb
 #include "xkb_glfw.h"
-#include "xkb_unicode.h"
 #include "egl_context.h"
 #include "osmesa_context.h"
 
@@ -204,10 +202,7 @@ typedef struct _GLFWlibraryWayland
     int32_t                     keyboardRepeatRate;
     int32_t                     keyboardRepeatDelay;
     struct {
-        long                    codepoint;
-        int                     plain;
-        int                     glfwKeyCode;
-        int                     scancode;
+        uint32_t                key;
         double                  nextRepeatAt;
         _GLFWwindow*            keyboardFocus;
     } keyRepeatInfo;

glfw/wl_window.c

@@ -693,10 +693,8 @@ static void
 dispatchPendingKeyRepeats() {
     if (_glfw.wl.keyRepeatInfo.nextRepeatAt <= 0 || _glfw.wl.keyRepeatInfo.keyboardFocus != _glfw.wl.keyboardFocus || _glfw.wl.keyboardRepeatRate == 0) return;
     double now = glfwGetTime();
-    const int mods = _glfw.wl.xkb.modifiers;
     while (_glfw.wl.keyRepeatInfo.nextRepeatAt <= now) {
-        _glfwInputKey(_glfw.wl.keyRepeatInfo.keyboardFocus, _glfw.wl.keyRepeatInfo.glfwKeyCode, _glfw.wl.keyRepeatInfo.scancode, GLFW_REPEAT, mods);
-        if (_glfw.wl.keyRepeatInfo.codepoint > -1) _glfwInputChar(_glfw.wl.keyRepeatInfo.keyboardFocus, _glfw.wl.keyRepeatInfo.codepoint, mods, _glfw.wl.keyRepeatInfo.plain);
+        glfw_xkb_handle_key_event(_glfw.wl.keyRepeatInfo.keyboardFocus, &_glfw.wl.xkb, _glfw.wl.keyRepeatInfo.key, GLFW_REPEAT);
         _glfw.wl.keyRepeatInfo.nextRepeatAt += 1.0 / _glfw.wl.keyboardRepeatRate;
         now = glfwGetTime();
     }
@@ -1244,13 +1242,12 @@ void _glfwPlatformSetCursorMode(_GLFWwindow* window, int mode)
 
 const char* _glfwPlatformGetScancodeName(int scancode)
 {
-    // TODO
-    return NULL;
+    return glfw_xkb_keysym_name(scancode);
 }
 
 int _glfwPlatformGetKeyScancode(int key)
 {
-    return _glfw.wl.xkb.scancodes[key];
+    return glfw_xkb_sym_for_key(key);
 }
 
 int _glfwPlatformCreateCursor(_GLFWcursor* cursor,

glfw/x11_init.c

@@ -36,319 +36,6 @@
 #include <locale.h>
 
 
-// Translate an X11 key code to a GLFW key code.
-//
-static int translateKeyCode(int scancode)
-{
-    int keySym;
-
-    // Valid key code range is  [8,255], according to the Xlib manual
-    if (scancode < 8 || scancode > 255)
-        return GLFW_KEY_UNKNOWN;
-
-    if (_glfw.x11.xkb.available)
-    {
-        // Try secondary keysym, for numeric keypad keys
-        // Note: This way we always force "NumLock = ON", which is intentional
-        // since the returned key code should correspond to a physical
-        // location.
-        keySym = XkbKeycodeToKeysym(_glfw.x11.display, scancode, 0, 1);
-        switch (keySym)
-        {
-            case XK_KP_0:           return GLFW_KEY_KP_0;
-            case XK_KP_1:           return GLFW_KEY_KP_1;
-            case XK_KP_2:           return GLFW_KEY_KP_2;
-            case XK_KP_3:           return GLFW_KEY_KP_3;
-            case XK_KP_4:           return GLFW_KEY_KP_4;
-            case XK_KP_5:           return GLFW_KEY_KP_5;
-            case XK_KP_6:           return GLFW_KEY_KP_6;
-            case XK_KP_7:           return GLFW_KEY_KP_7;
-            case XK_KP_8:           return GLFW_KEY_KP_8;
-            case XK_KP_9:           return GLFW_KEY_KP_9;
-            case XK_KP_Separator:
-            case XK_KP_Decimal:     return GLFW_KEY_KP_DECIMAL;
-            case XK_KP_Equal:       return GLFW_KEY_KP_EQUAL;
-            case XK_KP_Enter:       return GLFW_KEY_KP_ENTER;
-            default:                break;
-        }
-
-        // Now try primary keysym for function keys (non-printable keys)
-        // These should not depend on the current keyboard layout
-        keySym = XkbKeycodeToKeysym(_glfw.x11.display, scancode, 0, 0);
-    }
-    else
-    {
-        int dummy;
-        KeySym* keySyms;
-
-        keySyms = XGetKeyboardMapping(_glfw.x11.display, scancode, 1, &dummy);
-        keySym = keySyms[0];
-        XFree(keySyms);
-    }
-
-    switch (keySym)
-    {
-        case XK_Escape:         return GLFW_KEY_ESCAPE;
-        case XK_Tab:            return GLFW_KEY_TAB;
-        case XK_Shift_L:        return GLFW_KEY_LEFT_SHIFT;
-        case XK_Shift_R:        return GLFW_KEY_RIGHT_SHIFT;
-        case XK_Control_L:      return GLFW_KEY_LEFT_CONTROL;
-        case XK_Control_R:      return GLFW_KEY_RIGHT_CONTROL;
-        case XK_Meta_L:
-        case XK_Alt_L:          return GLFW_KEY_LEFT_ALT;
-        case XK_Mode_switch: // Mapped to Alt_R on many keyboards
-        case XK_ISO_Level3_Shift: // AltGr on at least some machines
-        case XK_Meta_R:
-        case XK_Alt_R:          return GLFW_KEY_RIGHT_ALT;
-        case XK_Super_L:        return GLFW_KEY_LEFT_SUPER;
-        case XK_Super_R:        return GLFW_KEY_RIGHT_SUPER;
-        case XK_Menu:           return GLFW_KEY_MENU;
-        case XK_Num_Lock:       return GLFW_KEY_NUM_LOCK;
-        case XK_Caps_Lock:      return GLFW_KEY_CAPS_LOCK;
-        case XK_Print:          return GLFW_KEY_PRINT_SCREEN;
-        case XK_Scroll_Lock:    return GLFW_KEY_SCROLL_LOCK;
-        case XK_Pause:          return GLFW_KEY_PAUSE;
-        case XK_Delete:         return GLFW_KEY_DELETE;
-        case XK_BackSpace:      return GLFW_KEY_BACKSPACE;
-        case XK_Return:         return GLFW_KEY_ENTER;
-        case XK_Home:           return GLFW_KEY_HOME;
-        case XK_End:            return GLFW_KEY_END;
-        case XK_Page_Up:        return GLFW_KEY_PAGE_UP;
-        case XK_Page_Down:      return GLFW_KEY_PAGE_DOWN;
-        case XK_Insert:         return GLFW_KEY_INSERT;
-        case XK_Left:           return GLFW_KEY_LEFT;
-        case XK_Right:          return GLFW_KEY_RIGHT;
-        case XK_Down:           return GLFW_KEY_DOWN;
-        case XK_Up:             return GLFW_KEY_UP;
-        case XK_F1:             return GLFW_KEY_F1;
-        case XK_F2:             return GLFW_KEY_F2;
-        case XK_F3:             return GLFW_KEY_F3;
-        case XK_F4:             return GLFW_KEY_F4;
-        case XK_F5:             return GLFW_KEY_F5;
-        case XK_F6:             return GLFW_KEY_F6;
-        case XK_F7:             return GLFW_KEY_F7;
-        case XK_F8:             return GLFW_KEY_F8;
-        case XK_F9:             return GLFW_KEY_F9;
-        case XK_F10:            return GLFW_KEY_F10;
-        case XK_F11:            return GLFW_KEY_F11;
-        case XK_F12:            return GLFW_KEY_F12;
-        case XK_F13:            return GLFW_KEY_F13;
-        case XK_F14:            return GLFW_KEY_F14;
-        case XK_F15:            return GLFW_KEY_F15;
-        case XK_F16:            return GLFW_KEY_F16;
-        case XK_F17:            return GLFW_KEY_F17;
-        case XK_F18:            return GLFW_KEY_F18;
-        case XK_F19:            return GLFW_KEY_F19;
-        case XK_F20:            return GLFW_KEY_F20;
-        case XK_F21:            return GLFW_KEY_F21;
-        case XK_F22:            return GLFW_KEY_F22;
-        case XK_F23:            return GLFW_KEY_F23;
-        case XK_F24:            return GLFW_KEY_F24;
-        case XK_F25:            return GLFW_KEY_F25;
-
-        // Numeric keypad
-        case XK_KP_Divide:      return GLFW_KEY_KP_DIVIDE;
-        case XK_KP_Multiply:    return GLFW_KEY_KP_MULTIPLY;
-        case XK_KP_Subtract:    return GLFW_KEY_KP_SUBTRACT;
-        case XK_KP_Add:         return GLFW_KEY_KP_ADD;
-
-        // These should have been detected in secondary keysym test above!
-        case XK_KP_Insert:      return GLFW_KEY_KP_0;
-        case XK_KP_End:         return GLFW_KEY_KP_1;
-        case XK_KP_Down:        return GLFW_KEY_KP_2;
-        case XK_KP_Page_Down:   return GLFW_KEY_KP_3;
-        case XK_KP_Left:        return GLFW_KEY_KP_4;
-        case XK_KP_Right:       return GLFW_KEY_KP_6;
-        case XK_KP_Home:        return GLFW_KEY_KP_7;
-        case XK_KP_Up:          return GLFW_KEY_KP_8;
-        case XK_KP_Page_Up:     return GLFW_KEY_KP_9;
-        case XK_KP_Delete:      return GLFW_KEY_KP_DECIMAL;
-        case XK_KP_Equal:       return GLFW_KEY_KP_EQUAL;
-        case XK_KP_Enter:       return GLFW_KEY_KP_ENTER;
-
-        // Last resort: Check for printable keys (should not happen if the XKB
-        // extension is available). This will give a layout dependent mapping
-        // (which is wrong, and we may miss some keys, especially on non-US
-        // keyboards), but it's better than nothing...
-        case XK_a:              return GLFW_KEY_A;
-        case XK_b:              return GLFW_KEY_B;
-        case XK_c:              return GLFW_KEY_C;
-        case XK_d:              return GLFW_KEY_D;
-        case XK_e:              return GLFW_KEY_E;
-        case XK_f:              return GLFW_KEY_F;
-        case XK_g:              return GLFW_KEY_G;
-        case XK_h:              return GLFW_KEY_H;
-        case XK_i:              return GLFW_KEY_I;
-        case XK_j:              return GLFW_KEY_J;
-        case XK_k:              return GLFW_KEY_K;
-        case XK_l:              return GLFW_KEY_L;
-        case XK_m:              return GLFW_KEY_M;
-        case XK_n:              return GLFW_KEY_N;
-        case XK_o:              return GLFW_KEY_O;
-        case XK_p:              return GLFW_KEY_P;
-        case XK_q:              return GLFW_KEY_Q;
-        case XK_r:              return GLFW_KEY_R;
-        case XK_s:              return GLFW_KEY_S;
-        case XK_t:              return GLFW_KEY_T;
-        case XK_u:              return GLFW_KEY_U;
-        case XK_v:              return GLFW_KEY_V;
-        case XK_w:              return GLFW_KEY_W;
-        case XK_x:              return GLFW_KEY_X;
-        case XK_y:              return GLFW_KEY_Y;
-        case XK_z:              return GLFW_KEY_Z;
-        case XK_1:              return GLFW_KEY_1;
-        case XK_2:              return GLFW_KEY_2;
-        case XK_3:              return GLFW_KEY_3;
-        case XK_4:              return GLFW_KEY_4;
-        case XK_5:              return GLFW_KEY_5;
-        case XK_6:              return GLFW_KEY_6;
-        case XK_7:              return GLFW_KEY_7;
-        case XK_8:              return GLFW_KEY_8;
-        case XK_9:              return GLFW_KEY_9;
-        case XK_0:              return GLFW_KEY_0;
-        case XK_space:          return GLFW_KEY_SPACE;
-        case XK_minus:          return GLFW_KEY_MINUS;
-        case XK_equal:          return GLFW_KEY_EQUAL;
-        case XK_bracketleft:    return GLFW_KEY_LEFT_BRACKET;
-        case XK_bracketright:   return GLFW_KEY_RIGHT_BRACKET;
-        case XK_backslash:      return GLFW_KEY_BACKSLASH;
-        case XK_semicolon:      return GLFW_KEY_SEMICOLON;
-        case XK_apostrophe:     return GLFW_KEY_APOSTROPHE;
-        case XK_grave:          return GLFW_KEY_GRAVE_ACCENT;
-        case XK_comma:          return GLFW_KEY_COMMA;
-        case XK_period:         return GLFW_KEY_PERIOD;
-        case XK_slash:          return GLFW_KEY_SLASH;
-        case XK_less:           return GLFW_KEY_WORLD_1; // At least in some layouts...
-        default:                break;
-    }
-
-    // No matching translation was found
-    return GLFW_KEY_UNKNOWN;
-}
-
-// Create key code translation tables
-//
-static void createKeyTables(void)
-{
-    int scancode, key;
-
-    memset(_glfw.x11.keycodes, -1, sizeof(_glfw.x11.keycodes));
-    memset(_glfw.x11.scancodes, -1, sizeof(_glfw.x11.scancodes));
-
-    if (_glfw.x11.xkb.available)
-    {
-        // Use XKB to determine physical key locations independently of the
-        // current keyboard layout
-
-        char name[XkbKeyNameLength + 1];
-        XkbDescPtr desc = XkbGetMap(_glfw.x11.display, 0, XkbUseCoreKbd);
-        XkbGetNames(_glfw.x11.display, XkbKeyNamesMask, desc);
-
-        // Find the X11 key code -> GLFW key code mapping
-        for (scancode = desc->min_key_code;  scancode <= desc->max_key_code;  scancode++)
-        {
-            memcpy(name, desc->names->keys[scancode].name, XkbKeyNameLength);
-            name[XkbKeyNameLength] = '\0';
-
-            // Map the key name to a GLFW key code. Note: We only map printable
-            // keys here, and we use the US keyboard layout. The rest of the
-            // keys (function keys) are mapped using traditional KeySym
-            // translations.
-            if (strcmp(name, "TLDE") == 0) key = GLFW_KEY_GRAVE_ACCENT;
-            else if (strcmp(name, "AE01") == 0) key = GLFW_KEY_1;
-            else if (strcmp(name, "AE02") == 0) key = GLFW_KEY_2;
-            else if (strcmp(name, "AE03") == 0) key = GLFW_KEY_3;
-            else if (strcmp(name, "AE04") == 0) key = GLFW_KEY_4;
-            else if (strcmp(name, "AE05") == 0) key = GLFW_KEY_5;
-            else if (strcmp(name, "AE06") == 0) key = GLFW_KEY_6;
-            else if (strcmp(name, "AE07") == 0) key = GLFW_KEY_7;
-            else if (strcmp(name, "AE08") == 0) key = GLFW_KEY_8;
-            else if (strcmp(name, "AE09") == 0) key = GLFW_KEY_9;
-            else if (strcmp(name, "AE10") == 0) key = GLFW_KEY_0;
-            else if (strcmp(name, "AE11") == 0) key = GLFW_KEY_MINUS;
-            else if (strcmp(name, "AE12") == 0) key = GLFW_KEY_EQUAL;
-            else if (strcmp(name, "AD01") == 0) key = GLFW_KEY_Q;
-            else if (strcmp(name, "AD02") == 0) key = GLFW_KEY_W;
-            else if (strcmp(name, "AD03") == 0) key = GLFW_KEY_E;
-            else if (strcmp(name, "AD04") == 0) key = GLFW_KEY_R;
-            else if (strcmp(name, "AD05") == 0) key = GLFW_KEY_T;
-            else if (strcmp(name, "AD06") == 0) key = GLFW_KEY_Y;
-            else if (strcmp(name, "AD07") == 0) key = GLFW_KEY_U;
-            else if (strcmp(name, "AD08") == 0) key = GLFW_KEY_I;
-            else if (strcmp(name, "AD09") == 0) key = GLFW_KEY_O;
-            else if (strcmp(name, "AD10") == 0) key = GLFW_KEY_P;
-            else if (strcmp(name, "AD11") == 0) key = GLFW_KEY_LEFT_BRACKET;
-            else if (strcmp(name, "AD12") == 0) key = GLFW_KEY_RIGHT_BRACKET;
-            else if (strcmp(name, "AC01") == 0) key = GLFW_KEY_A;
-            else if (strcmp(name, "AC02") == 0) key = GLFW_KEY_S;
-            else if (strcmp(name, "AC03") == 0) key = GLFW_KEY_D;
-            else if (strcmp(name, "AC04") == 0) key = GLFW_KEY_F;
-            else if (strcmp(name, "AC05") == 0) key = GLFW_KEY_G;
-            else if (strcmp(name, "AC06") == 0) key = GLFW_KEY_H;
-            else if (strcmp(name, "AC07") == 0) key = GLFW_KEY_J;
-            else if (strcmp(name, "AC08") == 0) key = GLFW_KEY_K;
-            else if (strcmp(name, "AC09") == 0) key = GLFW_KEY_L;
-            else if (strcmp(name, "AC10") == 0) key = GLFW_KEY_SEMICOLON;
-            else if (strcmp(name, "AC11") == 0) key = GLFW_KEY_APOSTROPHE;
-            else if (strcmp(name, "AB01") == 0) key = GLFW_KEY_Z;
-            else if (strcmp(name, "AB02") == 0) key = GLFW_KEY_X;
-            else if (strcmp(name, "AB03") == 0) key = GLFW_KEY_C;
-            else if (strcmp(name, "AB04") == 0) key = GLFW_KEY_V;
-            else if (strcmp(name, "AB05") == 0) key = GLFW_KEY_B;
-            else if (strcmp(name, "AB06") == 0) key = GLFW_KEY_N;
-            else if (strcmp(name, "AB07") == 0) key = GLFW_KEY_M;
-            else if (strcmp(name, "AB08") == 0) key = GLFW_KEY_COMMA;
-            else if (strcmp(name, "AB09") == 0) key = GLFW_KEY_PERIOD;
-            else if (strcmp(name, "AB10") == 0) key = GLFW_KEY_SLASH;
-            else if (strcmp(name, "BKSL") == 0) key = GLFW_KEY_BACKSLASH;
-            else if (strcmp(name, "LSGT") == 0) key = GLFW_KEY_WORLD_1;
-            else key = GLFW_KEY_UNKNOWN;
-
-            if ((scancode >= 0) && (scancode < 256))
-                _glfw.x11.keycodes[scancode] = key;
-        }
-
-        XkbFreeNames(desc, XkbKeyNamesMask, True);
-        XkbFreeKeyboard(desc, 0, True);
-    }
-
-    for (scancode = 0;  scancode < 256;  scancode++)
-    {
-        // Translate the un-translated key codes using traditional X11 KeySym
-        // lookups
-        if (_glfw.x11.keycodes[scancode] < 0)
-            _glfw.x11.keycodes[scancode] = translateKeyCode(scancode);
-
-        // Store the reverse translation for faster key name lookup
-        if (_glfw.x11.keycodes[scancode] > 0)
-            _glfw.x11.scancodes[_glfw.x11.keycodes[scancode]] = scancode;
-    }
-}
-
-// Check whether the IM has a usable style
-//
-static GLFWbool hasUsableInputMethodStyle(void)
-{
-    unsigned int i;
-    GLFWbool found = GLFW_FALSE;
-    XIMStyles* styles = NULL;
-
-    if (XGetIMValues(_glfw.x11.im, XNQueryInputStyle, &styles, NULL) != NULL)
-        return GLFW_FALSE;
-
-    for (i = 0;  i < styles->count_styles;  i++)
-    {
-        if (styles->supported_styles[i] == (XIMPreeditNothing | XIMStatusNothing))
-        {
-            found = GLFW_TRUE;
-            break;
-        }
-    }
-
-    XFree(styles);
-    return found;
-}
-
 // Check whether the specified atom is supported
 //
 static Atom getSupportedAtom(Atom* supportedAtoms,
@@ -479,7 +166,11 @@ static GLFWbool initExtensions(void)
                                       &_glfw.x11.vidmode.errorBase);
     }
 
+#if defined(__CYGWIN__)
+    _glfw.x11.xi.handle = _glfw_dlopen("libXi-6.so");
+#else
     _glfw.x11.xi.handle = _glfw_dlopen("libXi.so.6");
+#endif
     if (_glfw.x11.xi.handle)
     {
         _glfw.x11.xi.QueryVersion = (PFN_XIQueryVersion)
@@ -505,7 +196,11 @@ static GLFWbool initExtensions(void)
         }
     }
 
+#if defined(__CYGWIN__)
+    _glfw.x11.randr.handle = _glfw_dlopen("libXrandr-2.so");
+#else
     _glfw.x11.randr.handle = _glfw_dlopen("libXrandr.so.2");
+#endif
     if (_glfw.x11.randr.handle)
     {
         _glfw.x11.randr.AllocGamma = (PFN_XRRAllocGamma)
@@ -593,7 +288,11 @@ static GLFWbool initExtensions(void)
                        RROutputChangeNotifyMask);
     }
 
+#if defined(__CYGWIN__)
+    _glfw.x11.xcursor.handle = _glfw_dlopen("libXcursor-1.so");
+#else
     _glfw.x11.xcursor.handle = _glfw_dlopen("libXcursor.so.1");
+#endif
     if (_glfw.x11.xcursor.handle)
     {
         _glfw.x11.xcursor.ImageCreate = (PFN_XcursorImageCreate)
@@ -604,7 +303,11 @@ static GLFWbool initExtensions(void)
             _glfw_dlsym(_glfw.x11.xcursor.handle, "XcursorImageLoadCursor");
     }
 
+#if defined(__CYGWIN__)
+    _glfw.x11.xinerama.handle = _glfw_dlopen("libXinerama-1.so");
+#else
     _glfw.x11.xinerama.handle = _glfw_dlopen("libXinerama.so.1");
+#endif
     if (_glfw.x11.xinerama.handle)
     {
         _glfw.x11.xinerama.IsActive = (PFN_XineramaIsActive)
@@ -623,35 +326,11 @@ static GLFWbool initExtensions(void)
         }
     }
 
-    _glfw.x11.xkb.major = 1;
-    _glfw.x11.xkb.minor = 0;
-    _glfw.x11.xkb.available =
-        XkbQueryExtension(_glfw.x11.display,
-                          &_glfw.x11.xkb.majorOpcode,
-                          &_glfw.x11.xkb.eventBase,
-                          &_glfw.x11.xkb.errorBase,
-                          &_glfw.x11.xkb.major,
-                          &_glfw.x11.xkb.minor);
-
-    if (_glfw.x11.xkb.available)
-    {
-        Bool supported;
-
-        if (XkbSetDetectableAutoRepeat(_glfw.x11.display, True, &supported))
-        {
-            if (supported)
-                _glfw.x11.xkb.detectable = GLFW_TRUE;
-        }
-    }
-
-    _glfw.x11.x11xcb.handle = _glfw_dlopen("libX11-xcb.so.1");
-    if (_glfw.x11.x11xcb.handle)
-    {
-        _glfw.x11.x11xcb.GetXCBConnection = (PFN_XGetXCBConnection)
-            _glfw_dlsym(_glfw.x11.x11xcb.handle, "XGetXCBConnection");
-    }
-
+#if defined(__CYGWIN__)
+    _glfw.x11.xrender.handle = _glfw_dlopen("libXrender-1.so");
+#else
     _glfw.x11.xrender.handle = _glfw_dlopen("libXrender.so.1");
+#endif
     if (_glfw.x11.xrender.handle)
     {
         _glfw.x11.xrender.QueryExtension = (PFN_XRenderQueryExtension)
@@ -674,10 +353,31 @@ static GLFWbool initExtensions(void)
         }
     }
 
-    // Update the key code LUT
-    // FIXME: We should listen to XkbMapNotify events to track changes to
-    // the keyboard mapping.
-    createKeyTables();
+    _glfw.x11.xkb.major = 1;
+    _glfw.x11.xkb.minor = 0;
+    _glfw.x11.xkb.available = XkbQueryExtension(_glfw.x11.display,
+            &_glfw.x11.xkb.majorOpcode,
+            &_glfw.x11.xkb.eventBase,
+            &_glfw.x11.xkb.errorBase,
+            &_glfw.x11.xkb.major,
+            &_glfw.x11.xkb.minor);
+
+    if (!_glfw.x11.xkb.available)
+    {
+        _glfwInputError(GLFW_PLATFORM_ERROR, "X11: Failed to load Xkb extension");
+        return GLFW_FALSE;
+    }
+    Bool supported;
+    if (XkbSetDetectableAutoRepeat(_glfw.x11.display, True, &supported))
+    {
+        if (supported)
+            _glfw.x11.xkb.detectable = GLFW_TRUE;
+    }
+
+    if (!glfw_xkb_set_x11_events_mask()) return GLFW_FALSE;
+    if (!glfw_xkb_create_context(&_glfw.x11.xkb)) return GLFW_FALSE;
+    if (!glfw_xkb_update_x11_keyboard_id(&_glfw.x11.xkb)) return GLFW_FALSE;
+    if (!glfw_xkb_compile_keymap(&_glfw.x11.xkb, NULL)) return GLFW_FALSE;
 
     // Detect whether an EWMH-conformant window manager is running
     detectEWMH();
@@ -896,16 +596,6 @@ Cursor _glfwCreateCursorX11(const GLFWimage* image, int xhot, int yhot)
 
 int _glfwPlatformInit(void)
 {
-#if !defined(X_HAVE_UTF8_STRING)
-    // HACK: If the current locale is "C" and the Xlib UTF-8 functions are
-    //       unavailable, apply the environment's locale in the hope that it's
-    //       both available and not "C"
-    //       This is done because the "C" locale breaks wide character input,
-    //       which is what we fall back on when UTF-8 support is missing
-    if (strcmp(setlocale(LC_CTYPE, NULL), "C") == 0)
-        setlocale(LC_CTYPE, "");
-#endif
-
     XInitThreads();
     XrmInitialize();
 
@@ -939,21 +629,6 @@ int _glfwPlatformInit(void)
     _glfw.x11.helperWindowHandle = createHelperWindow();
     _glfw.x11.hiddenCursorHandle = createHiddenCursor();
 
-    if (XSupportsLocale())
-    {
-        XSetLocaleModifiers("");
-
-        _glfw.x11.im = XOpenIM(_glfw.x11.display, 0, NULL, NULL);
-        if (_glfw.x11.im)
-        {
-            if (!hasUsableInputMethodStyle())
-            {
-                XCloseIM(_glfw.x11.im);
-                _glfw.x11.im = NULL;
-            }
-        }
-    }
-
 #if defined(__linux__)
     if (!_glfwInitJoysticksLinux())
         return GLFW_FALSE;
@@ -985,27 +660,16 @@ void _glfwPlatformTerminate(void)
         _glfw.x11.hiddenCursorHandle = (Cursor) 0;
     }
 
+    glfw_xkb_release(&_glfw.x11.xkb);
     free(_glfw.x11.primarySelectionString);
     free(_glfw.x11.clipboardString);
 
-    if (_glfw.x11.im)
-    {
-        XCloseIM(_glfw.x11.im);
-        _glfw.x11.im = NULL;
-    }
-
     if (_glfw.x11.display)
     {
         XCloseDisplay(_glfw.x11.display);
         _glfw.x11.display = NULL;
     }
 
-    if (_glfw.x11.x11xcb.handle)
-    {
-        _glfw_dlclose(_glfw.x11.x11xcb.handle);
-        _glfw.x11.x11xcb.handle = NULL;
-    }
-
     if (_glfw.x11.xcursor.handle)
     {
         _glfw_dlclose(_glfw.x11.xcursor.handle);
@@ -1024,6 +688,24 @@ void _glfwPlatformTerminate(void)
         _glfw.x11.xinerama.handle = NULL;
     }
 
+    if (_glfw.x11.xrender.handle)
+    {
+        _glfw_dlclose(_glfw.x11.xrender.handle);
+        _glfw.x11.xrender.handle = NULL;
+    }
+
+    if (_glfw.x11.vidmode.handle)
+    {
+        _glfw_dlclose(_glfw.x11.vidmode.handle);
+        _glfw.x11.vidmode.handle = NULL;
+    }
+
+    if (_glfw.x11.xi.handle)
+    {
+        _glfw_dlclose(_glfw.x11.xi.handle);
+        _glfw.x11.xi.handle = NULL;
+    }
+
     // NOTE: These need to be unloaded after XCloseDisplay, as they register
     //       cleanup callbacks that get called by that function
     _glfwTerminateEGL();
@@ -1050,4 +732,3 @@ const char* _glfwPlatformGetVersionString(void)
 #endif
         ;
 }
-

glfw/x11_platform.h

@@ -35,6 +35,9 @@
 #include <X11/Xatom.h>
 #include <X11/Xcursor/Xcursor.h>
 
+// The xcb library is needed to work with libxkb
+#include <X11/Xlib-xcb.h>
+
 // The XRandR extension provides mode setting and gamma control
 #include <X11/extensions/Xrandr.h>
 
@@ -47,6 +50,9 @@
 // The XInput extension provides raw mouse motion input
 #include <X11/extensions/XInput2.h>
 
+// The libxkb library is used for improved keyboard support
+#include "xkb_glfw.h"
+
 typedef XRRCrtcGamma* (* PFN_XRRAllocGamma)(int);
 typedef void (* PFN_XRRFreeCrtcInfo)(XRRCrtcInfo*);
 typedef void (* PFN_XRRFreeGamma)(XRRCrtcGamma*);
@@ -96,11 +102,7 @@ typedef XineramaScreenInfo* (* PFN_XineramaQueryScreens)(Display*,int*);
 #define XineramaQueryExtension _glfw.x11.xinerama.QueryExtension
 #define XineramaQueryScreens _glfw.x11.xinerama.QueryScreens
 
-typedef XID xcb_window_t;
-typedef XID xcb_visualid_t;
 typedef struct xcb_connection_t xcb_connection_t;
-typedef xcb_connection_t* (* PFN_XGetXCBConnection)(Display*);
-#define XGetXCBConnection _glfw.x11.x11xcb.GetXCBConnection
 
 typedef Bool (* PFN_XF86VidModeQueryExtension)(Display*,int*,int*);
 typedef Bool (* PFN_XF86VidModeGetGammaRamp)(Display*,int,int,unsigned short*,unsigned short*,unsigned short*);
@@ -151,7 +153,6 @@ typedef VkBool32 (APIENTRY *PFN_vkGetPhysicalDeviceXcbPresentationSupportKHR)(Vk
 
 #include "posix_thread.h"
 #include "posix_time.h"
-#include "xkb_unicode.h"
 #include "glx_context.h"
 #include "egl_context.h"
 #include "osmesa_context.h"
@@ -180,7 +181,6 @@ typedef struct _GLFWwindowX11
 {
     Colormap        colormap;
     Window          handle;
-    XIC             ic;
 
     GLFWbool        overrideRedirect;
     GLFWbool        iconified;
@@ -219,20 +219,12 @@ typedef struct _GLFWlibraryX11
     Cursor          hiddenCursorHandle;
     // Context for mapping window XIDs to _GLFWwindow pointers
     XContext        context;
-    // XIM input method
-    XIM             im;
     // Most recent error code received by X error handler
     int             errorCode;
     // Primary selection string (while the primary selection is owned)
     char*           primarySelectionString;
     // Clipboard string (while the selection is owned)
     char*           clipboardString;
-    // Key name string
-    char            keyName[5];
-    // X11 keycode to GLFW key LUT
-    short int       keycodes[256];
-    // GLFW key to X11 keycode LUT
-    short int       scancodes[GLFW_KEY_LAST + 1];
     // Where to place the cursor when re-enabled
     double          restoreCursorPosX, restoreCursorPosY;
     // The window whose disabled cursor mode is active
@@ -318,15 +310,7 @@ typedef struct _GLFWlibraryX11
         PFN_XRRUpdateConfiguration UpdateConfiguration;
     } randr;
 
-    struct {
-        GLFWbool    available;
-        GLFWbool    detectable;
-        int         majorOpcode;
-        int         eventBase;
-        int         errorBase;
-        int         major;
-        int         minor;
-    } xkb;
+    _GLFWXKBData xkb;
 
     struct {
         int         count;
@@ -359,11 +343,6 @@ typedef struct _GLFWlibraryX11
         PFN_XineramaQueryScreens QueryScreens;
     } xinerama;
 
-    struct {
-        void*       handle;
-        PFN_XGetXCBConnection GetXCBConnection;
-    } x11xcb;
-
     struct {
         GLFWbool    available;
         void*       handle;
@@ -441,4 +420,3 @@ void _glfwReleaseErrorHandlerX11(void);
 void _glfwInputErrorX11(int error, const char* message);
 
 void _glfwPushSelectionToManagerX11(void);
-

glfw/x11_window.c

@@ -220,17 +220,6 @@ static int translateState(int state)
     return mods;
 }
 
-// Translates an X11 key code to a GLFW key token
-//
-static int translateKey(int scancode)
-{
-    // Use the pre-filled LUT (see createKeyTables() in x11_init.c)
-    if (scancode < 0 || scancode > 255)
-        return GLFW_KEY_UNKNOWN;
-
-    return _glfw.x11.keycodes[scancode];
-}
-
 // Return the GLFW window corresponding to the specified X11 window
 //
 static _GLFWwindow* findWindowByHandle(Window handle)
@@ -497,31 +486,6 @@ static size_t encodeUTF8(char* s, unsigned int ch)
     return count;
 }
 
-// Decode a Unicode code point from a UTF-8 stream
-// Based on cutef8 by Jeff Bezanson (Public Domain)
-//
-#if defined(X_HAVE_UTF8_STRING)
-static unsigned int decodeUTF8(const char** s)
-{
-    unsigned int ch = 0, count = 0;
-    static const unsigned int offsets[] =
-    {
-        0x00000000u, 0x00003080u, 0x000e2080u,
-        0x03c82080u, 0xfa082080u, 0x82082080u
-    };
-
-    do
-    {
-        ch = (ch << 6) + (unsigned char) **s;
-        (*s)++;
-        count++;
-    } while ((**s & 0xc0) == 0x80);
-
-    assert(count <= 6);
-    return ch - offsets[count - 1];
-}
-#endif /*X_HAVE_UTF8_STRING*/
-
 // Convert the specified Latin-1 string to UTF-8
 //
 static char* convertLatin1toUTF8(const char* source)
@@ -571,6 +535,61 @@ static void updateCursorImage(_GLFWwindow* window)
     }
 }
 
+// Apply disabled cursor mode to a focused window
+//
+static void disableCursor(_GLFWwindow* window)
+{
+    if (_glfw.x11.xi.available)
+    {
+        XIEventMask em;
+        unsigned char mask[XIMaskLen(XI_RawMotion)] = { 0 };
+
+        em.deviceid = XIAllMasterDevices;
+        em.mask_len = sizeof(mask);
+        em.mask = mask;
+        XISetMask(mask, XI_RawMotion);
+
+        XISelectEvents(_glfw.x11.display, _glfw.x11.root, &em, 1);
+    }
+
+    _glfw.x11.disabledCursorWindow = window;
+    _glfwPlatformGetCursorPos(window,
+                              &_glfw.x11.restoreCursorPosX,
+                              &_glfw.x11.restoreCursorPosY);
+    updateCursorImage(window);
+    centerCursor(window);
+    XGrabPointer(_glfw.x11.display, window->x11.handle, True,
+                 ButtonPressMask | ButtonReleaseMask | PointerMotionMask,
+                 GrabModeAsync, GrabModeAsync,
+                 window->x11.handle,
+                 _glfw.x11.hiddenCursorHandle,
+                 CurrentTime);
+}
+
+// Exit disabled cursor mode for the specified window
+//
+static void enableCursor(_GLFWwindow* window)
+{
+    if (_glfw.x11.xi.available)
+    {
+        XIEventMask em;
+        unsigned char mask[] = { 0 };
+
+        em.deviceid = XIAllMasterDevices;
+        em.mask_len = sizeof(mask);
+        em.mask = mask;
+
+        XISelectEvents(_glfw.x11.display, _glfw.x11.root, &em, 1);
+    }
+
+    _glfw.x11.disabledCursorWindow = NULL;
+    XUngrabPointer(_glfw.x11.display, CurrentTime);
+    _glfwPlatformSetCursorPos(window,
+                              _glfw.x11.restoreCursorPosX,
+                              _glfw.x11.restoreCursorPosY);
+    updateCursorImage(window);
+}
+
 // Create the X11 window (and its colormap)
 //
 static GLFWbool createNativeWindow(_GLFWwindow* window,
@@ -746,19 +765,6 @@ static GLFWbool createNativeWindow(_GLFWwindow* window,
     }
 
     _glfwPlatformSetWindowTitle(window, wndconfig->title);
-
-    if (_glfw.x11.im)
-    {
-        window->x11.ic = XCreateIC(_glfw.x11.im,
-                                   XNInputStyle,
-                                   XIMPreeditNothing | XIMStatusNothing,
-                                   XNClientWindow,
-                                   window->x11.handle,
-                                   XNFocusWindow,
-                                   window->x11.handle,
-                                   NULL);
-    }
-
     _glfwPlatformGetWindowPos(window, &window->x11.xpos, &window->x11.ypos);
     _glfwPlatformGetWindowSize(window, &window->x11.width, &window->x11.height);
 
@@ -1141,16 +1147,8 @@ static void releaseMonitor(_GLFWwindow* window)
 static void processEvent(XEvent *event)
 {
     _GLFWwindow* window = NULL;
-    int keycode = 0;
     Bool filtered = False;
 
-    // HACK: Save scancode as some IMs clear the field in XFilterEvent
-    if (event->type == KeyPress || event->type == KeyRelease)
-        keycode = event->xkey.keycode;
-
-    if (_glfw.x11.im)
-        filtered = XFilterEvent(event, None);
-
     if (_glfw.x11.randr.available)
     {
         if (event->type == _glfw.x11.randr.eventBase + RRNotify)
@@ -1208,6 +1206,30 @@ static void processEvent(XEvent *event)
         handleSelectionRequest(event);
         return;
     }
+    else if (event->type == _glfw.x11.xkb.eventBase)
+    {
+        XkbEvent *kb_event = (XkbEvent*)event;
+        switch(kb_event->any.xkb_type) {
+            case XkbNewKeyboardNotify:
+                if(!glfw_xkb_update_x11_keyboard_id(&_glfw.x11.xkb)) return;
+                /* fallthrough */
+            case XkbMapNotify:
+            {
+                glfw_xkb_compile_keymap(&_glfw.x11.xkb, NULL);
+                return;
+            }
+            case XkbStateNotify:
+            {
+                XkbStateNotifyEvent *state_event = (XkbStateNotifyEvent*)kb_event;
+                glfw_xkb_update_modifiers(
+                        &_glfw.x11.xkb, state_event->base_mods, state_event->latched_mods,
+                        state_event->locked_mods, state_event->base_group, state_event->latched_group, state_event->locked_group
+                );
+                return;
+            }
+        }
+        return;
+    }
 
     window = findWindowByHandle(event->xany.window);
     if (window == NULL)
@@ -1220,105 +1242,12 @@ static void processEvent(XEvent *event)
     {
         case KeyPress:
         {
-            const int key = translateKey(keycode);
-            const int mods = translateState(event->xkey.state);
-            const int plain = !(mods & (GLFW_MOD_CONTROL | GLFW_MOD_ALT));
-
-            if (window->x11.ic)
-            {
-                // HACK: Ignore duplicate key press events generated by ibus
-                //       These have the same timestamp as the original event
-                //       Corresponding release events are filtered out
-                //       implicitly by the GLFW key repeat logic
-                if (window->x11.lastKeyTime < event->xkey.time)
-                {
-                    if (keycode)
-                        _glfwInputKey(window, key, keycode, GLFW_PRESS, mods);
-
-                    window->x11.lastKeyTime = event->xkey.time;
-                }
-
-                if (!filtered)
-                {
-                    int count;
-                    Status status;
-#if defined(X_HAVE_UTF8_STRING)
-                    char buffer[100];
-                    char* chars = buffer;
-
-                    count = Xutf8LookupString(window->x11.ic,
-                                              &event->xkey,
-                                              buffer, sizeof(buffer) - 1,
-                                              NULL, &status);
-
-                    if (status == XBufferOverflow)
-                    {
-                        chars = calloc(count + 1, 1);
-                        count = Xutf8LookupString(window->x11.ic,
-                                                  &event->xkey,
-                                                  chars, count,
-                                                  NULL, &status);
-                    }
-
-                    if (status == XLookupChars || status == XLookupBoth)
-                    {
-                        const char* c = chars;
-                        chars[count] = '\0';
-                        while (c - chars < count)
-                            _glfwInputChar(window, decodeUTF8(&c), mods, plain);
-                    }
-#else /*X_HAVE_UTF8_STRING*/
-                    wchar_t buffer[16];
-                    wchar_t* chars = buffer;
-
-                    count = XwcLookupString(window->x11.ic,
-                                            &event->xkey,
-                                            buffer,
-                                            sizeof(buffer) / sizeof(wchar_t),
-                                            NULL,
-                                            &status);
-
-                    if (status == XBufferOverflow)
-                    {
-                        chars = calloc(count, sizeof(wchar_t));
-                        count = XwcLookupString(window->x11.ic,
-                                                &event->xkey,
-                                                chars, count,
-                                                NULL, &status);
-                    }
-
-                    if (status == XLookupChars || status == XLookupBoth)
-                    {
-                        int i;
-                        for (i = 0;  i < count;  i++)
-                            _glfwInputChar(window, chars[i], mods, plain);
-                    }
-#endif /*X_HAVE_UTF8_STRING*/
-
-                    if (chars != buffer)
-                        free(chars);
-                }
-            }
-            else
-            {
-                KeySym keysym;
-                XLookupString(&event->xkey, NULL, 0, &keysym, NULL);
-
-                _glfwInputKey(window, key, keycode, GLFW_PRESS, mods);
-
-                const long character = _glfwKeySym2Unicode(keysym);
-                if (character != -1)
-                    _glfwInputChar(window, character, mods, plain);
-            }
-
+            glfw_xkb_handle_key_event(window, &_glfw.x11.xkb, event->xkey.keycode, GLFW_PRESS);
             return;
         }
 
         case KeyRelease:
         {
-            const int key = translateKey(keycode);
-            const int mods = translateState(event->xkey.state);
-
             if (!_glfw.x11.xkb.detectable)
             {
                 // HACK: Key repeat events will arrive as KeyRelease/KeyPress
@@ -1332,7 +1261,7 @@ static void processEvent(XEvent *event)
 
                     if (next.type == KeyPress &&
                         next.xkey.window == event->xkey.window &&
-                        next.xkey.keycode == keycode)
+                        next.xkey.keycode == event->xkey.keycode)
                     {
                         // HACK: The time of repeat events sometimes doesn't
                         //       match that of the press event, so add an
@@ -1351,7 +1280,7 @@ static void processEvent(XEvent *event)
                 }
             }
 
-            _glfwInputKey(window, key, keycode, GLFW_RELEASE, mods);
+            glfw_xkb_handle_key_event(window, &_glfw.x11.xkb, event->xkey.keycode, GLFW_RELEASE);
             return;
         }
 
@@ -1432,7 +1361,7 @@ static void processEvent(XEvent *event)
             // HACK: This is a workaround for WMs (KWM, Fluxbox) that otherwise
             //       ignore the defined cursor for hidden cursor mode
             if (window->cursorMode == GLFW_CURSOR_HIDDEN)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_HIDDEN);
+                updateCursorImage(window);
 
             _glfwInputCursorEnter(window, GLFW_TRUE);
             return;
@@ -1725,7 +1654,7 @@ static void processEvent(XEvent *event)
         case FocusIn:
         {
             if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_DISABLED);
+                disableCursor(window);
 
             if (event->xfocus.mode == NotifyGrab ||
                 event->xfocus.mode == NotifyUngrab)
@@ -1735,9 +1664,6 @@ static void processEvent(XEvent *event)
                 return;
             }
 
-            if (window->x11.ic)
-                XSetICFocus(window->x11.ic);
-
             _glfwInputWindowFocus(window, GLFW_TRUE);
             return;
         }
@@ -1745,7 +1671,7 @@ static void processEvent(XEvent *event)
         case FocusOut:
         {
             if (window->cursorMode == GLFW_CURSOR_DISABLED)
-                _glfwPlatformSetCursorMode(window, GLFW_CURSOR_NORMAL);
+                enableCursor(window);
 
             if (event->xfocus.mode == NotifyGrab ||
                 event->xfocus.mode == NotifyUngrab)
@@ -1755,9 +1681,6 @@ static void processEvent(XEvent *event)
                 return;
             }
 
-            if (window->x11.ic)
-                XUnsetICFocus(window->x11.ic);
-
             if (window->monitor && window->autoIconify)
                 _glfwPlatformIconifyWindow(window);
 
@@ -1989,12 +1912,6 @@ void _glfwPlatformDestroyWindow(_GLFWwindow* window)
     if (window->monitor)
         releaseMonitor(window);
 
-    if (window->x11.ic)
-    {
-        XDestroyIC(window->x11.ic);
-        window->x11.ic = NULL;
-    }
-
     if (window->context.destroy)
         window->context.destroy(window);
 
@@ -2622,8 +2539,9 @@ void _glfwPlatformPollEvents(void)
 #if defined(__linux__)
     _glfwDetectJoystickConnectionLinux();
 #endif
-    int count = XPending(_glfw.x11.display);
-    while (count--)
+    XPending(_glfw.x11.display);
+
+    while (XQLength(_glfw.x11.display))
     {
         XEvent event;
         XNextEvent(_glfw.x11.display, &event);
@@ -2713,80 +2631,26 @@ void _glfwPlatformSetCursorMode(_GLFWwindow* window, int mode)
 {
     if (mode == GLFW_CURSOR_DISABLED)
     {
-        if (_glfw.x11.xi.available)
-        {
-            XIEventMask em;
-            unsigned char mask[XIMaskLen(XI_RawMotion)] = { 0 };
-
-            em.deviceid = XIAllMasterDevices;
-            em.mask_len = sizeof(mask);
-            em.mask = mask;
-            XISetMask(mask, XI_RawMotion);
-
-            XISelectEvents(_glfw.x11.display, _glfw.x11.root, &em, 1);
-        }
-
-        _glfw.x11.disabledCursorWindow = window;
-        _glfwPlatformGetCursorPos(window,
-                                  &_glfw.x11.restoreCursorPosX,
-                                  &_glfw.x11.restoreCursorPosY);
-        centerCursor(window);
-        XGrabPointer(_glfw.x11.display, window->x11.handle, True,
-                     ButtonPressMask | ButtonReleaseMask | PointerMotionMask,
-                     GrabModeAsync, GrabModeAsync,
-                     window->x11.handle,
-                     _glfw.x11.hiddenCursorHandle,
-                     CurrentTime);
+        if (_glfwPlatformWindowFocused(window))
+            disableCursor(window);
     }
     else if (_glfw.x11.disabledCursorWindow == window)
-    {
-        if (_glfw.x11.xi.available)
-        {
-            XIEventMask em;
-            unsigned char mask[] = { 0 };
+        enableCursor(window);
+    else
+        updateCursorImage(window);
 
-            em.deviceid = XIAllMasterDevices;
-            em.mask_len = sizeof(mask);
-            em.mask = mask;
-
-            XISelectEvents(_glfw.x11.display, _glfw.x11.root, &em, 1);
-        }
-
-        _glfw.x11.disabledCursorWindow = NULL;
-        XUngrabPointer(_glfw.x11.display, CurrentTime);
-        _glfwPlatformSetCursorPos(window,
-                                  _glfw.x11.restoreCursorPosX,
-                                  _glfw.x11.restoreCursorPosY);
-    }
-
-    updateCursorImage(window);
     XFlush(_glfw.x11.display);
 }
 
 const char* _glfwPlatformGetScancodeName(int scancode)
 {
-    if (!_glfw.x11.xkb.available)
-        return NULL;
 
-    const KeySym keysym = XkbKeycodeToKeysym(_glfw.x11.display, scancode, 0, 0);
-    if (keysym == NoSymbol)
-        return NULL;
-
-    const long ch = _glfwKeySym2Unicode(keysym);
-    if (ch == -1)
-        return NULL;
-
-    const size_t count = encodeUTF8(_glfw.x11.keyName, (unsigned int) ch);
-    if (count == 0)
-        return NULL;
-
-    _glfw.x11.keyName[count] = '\0';
-    return _glfw.x11.keyName;
+    return glfw_xkb_keysym_name(scancode);
 }
 
 int _glfwPlatformGetKeyScancode(int key)
 {
-    return _glfw.x11.scancodes[key];
+    return glfw_xkb_sym_for_key(key);
 }
 
 int _glfwPlatformCreateCursor(_GLFWcursor* cursor,
@@ -2857,7 +2721,7 @@ void _glfwPlatformGetRequiredInstanceExtensions(char** extensions)
     if (!_glfw.vk.KHR_surface)
         return;
 
-    if (!_glfw.vk.KHR_xcb_surface || !_glfw.x11.x11xcb.handle)
+    if (!_glfw.vk.KHR_xcb_surface)
     {
         if (!_glfw.vk.KHR_xlib_surface)
             return;
@@ -2867,7 +2731,7 @@ void _glfwPlatformGetRequiredInstanceExtensions(char** extensions)
 
     // NOTE: VK_KHR_xcb_surface is preferred due to some early ICDs exposing but
     //       not correctly implementing VK_KHR_xlib_surface
-    if (_glfw.vk.KHR_xcb_surface && _glfw.x11.x11xcb.handle)
+    if (_glfw.vk.KHR_xcb_surface)
         extensions[1] = "VK_KHR_xcb_surface";
     else
         extensions[1] = "VK_KHR_xlib_surface";
@@ -2880,7 +2744,7 @@ int _glfwPlatformGetPhysicalDevicePresentationSupport(VkInstance instance,
     VisualID visualID = XVisualIDFromVisual(DefaultVisual(_glfw.x11.display,
                                                           _glfw.x11.screen));
 
-    if (_glfw.vk.KHR_xcb_surface && _glfw.x11.x11xcb.handle)
+    if (_glfw.vk.KHR_xcb_surface)
     {
         PFN_vkGetPhysicalDeviceXcbPresentationSupportKHR
             vkGetPhysicalDeviceXcbPresentationSupportKHR =
@@ -2931,7 +2795,7 @@ VkResult _glfwPlatformCreateWindowSurface(VkInstance instance,
                                           const VkAllocationCallbacks* allocator,
                                           VkSurfaceKHR* surface)
 {
-    if (_glfw.vk.KHR_xcb_surface && _glfw.x11.x11xcb.handle)
+    if (_glfw.vk.KHR_xcb_surface)
     {
         VkResult err;
         VkXcbSurfaceCreateInfoKHR sci;

glfw/xkb_glfw.c

@@ -0,0 +1,427 @@
+//========================================================================
+// GLFW 3.3 XKB - www.glfw.org
+//------------------------------------------------------------------------
+// Copyright (c) 2018 Kovid Goyal <kovid@kovidgoyal.net>
+//
+// This software is provided 'as-is', without any express or implied
+// warranty. In no event will the authors be held liable for any damages
+// arising from the use of this software.
+//
+// Permission is granted to anyone to use this software for any purpose,
+// including commercial applications, and to alter it and redistribute it
+// freely, subject to the following restrictions:
+//
+// 1. The origin of this software must not be misrepresented; you must not
+//    claim that you wrote the original software. If you use this software
+//    in a product, an acknowledgment in the product documentation would
+//    be appreciated but is not required.
+//
+// 2. Altered source versions must be plainly marked as such, and must not
+//    be misrepresented as being the original software.
+//
+// 3. This notice may not be removed or altered from any source
+//    distribution.
+//
+//========================================================================
+
+
+#include <string.h>
+#include <stdlib.h>
+#include "internal.h"
+#include "xkb_glfw.h"
+
+static GLFWbool debug_keyboard = GLFW_FALSE;
+#define debug(...) if (debug_keyboard) printf(__VA_ARGS__);
+
+#define map_key(key) { \
+    switch(key) { \
+        S(space, SPACE); \
+        S(apostrophe, APOSTROPHE); \
+        S(comma, COMMA); \
+        S(minus, MINUS); \
+        S(period, PERIOD); \
+        S(slash, SLASH); \
+        S(semicolon, SEMICOLON); \
+        S(equal, EQUAL); \
+        S(bracketleft, LEFT_BRACKET); \
+        S(backslash, BACKSLASH); \
+        S(bracketright, RIGHT_BRACKET); \
+        S(grave, GRAVE_ACCENT); \
+        S(Escape, ESCAPE); \
+        S(Return, ENTER); \
+        S(Tab, TAB); \
+        S(BackSpace, BACKSPACE); \
+        S(Insert, INSERT); \
+        S(Delete, DELETE); \
+        S(Right, RIGHT); \
+        S(Left, LEFT); \
+        S(Up, UP); \
+        S(Down, DOWN); \
+        S(Page_Up, PAGE_UP); \
+        S(Page_Down, PAGE_DOWN); \
+        S(Home, HOME); \
+        S(End, END); \
+        S(Caps_Lock, CAPS_LOCK); \
+        S(Scroll_Lock, SCROLL_LOCK); \
+        S(Num_Lock, NUM_LOCK); \
+        S(Print, PRINT_SCREEN); \
+        S(Pause, PAUSE); \
+        S(KP_Decimal, KP_DECIMAL); \
+        S(KP_Divide, KP_DIVIDE); \
+        S(KP_Multiply, KP_MULTIPLY); \
+        S(KP_Subtract, KP_SUBTRACT); \
+        S(KP_Add, KP_ADD); \
+        S(KP_Enter, KP_ENTER); \
+        S(KP_Equal, KP_EQUAL); \
+        F(KP_Home, HOME); \
+        F(KP_End, END); \
+        F(KP_Page_Up, PAGE_UP); \
+        F(KP_Page_Down, PAGE_DOWN); \
+        F(KP_Insert, INSERT); \
+        F(KP_Delete, DELETE); \
+        S(Shift_L, LEFT_SHIFT); \
+        S(Control_L, LEFT_CONTROL); \
+        S(Alt_L, LEFT_ALT); \
+        S(Super_L, LEFT_SUPER); \
+        S(Shift_R, RIGHT_SHIFT); \
+        S(Control_R, RIGHT_CONTROL); \
+        S(Alt_R, RIGHT_ALT); \
+        S(Super_R, RIGHT_SUPER); \
+        S(Menu, MENU); \
+        R(0, 9, 0, 9); \
+        R(a, z, A, Z); \
+        D(A, Z, A, Z); \
+        R(F1, F25, F1, F25); \
+        R(KP_0, KP_9, KP_0, KP_9); \
+        default: \
+            break; \
+    } \
+}
+
+static int
+glfw_key_for_sym(xkb_keysym_t key) {
+#define S(f, t) case XKB_KEY_##f: return GLFW_KEY_##t
+#define F(f, t) S(f, t)
+#define R(s, e, gs, ...) case XKB_KEY_##s ... XKB_KEY_##e: return GLFW_KEY_##gs + key - XKB_KEY_##s
+#define D(s, e, gs, ...) R(s, e, gs)
+    map_key(key)
+    return GLFW_KEY_UNKNOWN;
+#undef F
+#undef D
+#undef R
+#undef S
+};
+
+xkb_keysym_t
+glfw_xkb_sym_for_key(int key) {
+#define S(f, t) case GLFW_KEY_##t: return XKB_KEY_##f
+#define F(...)
+#define R(s, e, gs, ge) case GLFW_KEY_##gs ... GLFW_KEY_##ge: return XKB_KEY_##s + key - GLFW_KEY_##gs
+#define D(...)
+    map_key(key)
+    return GLFW_KEY_UNKNOWN;
+#undef F
+#undef D
+#undef R
+#undef S
+}
+
+#ifdef _GLFW_X11
+
+GLFWbool
+glfw_xkb_set_x11_events_mask(void) {
+    if (!XkbSelectEvents(_glfw.x11.display, XkbUseCoreKbd, XkbNewKeyboardNotifyMask | XkbMapNotifyMask | XkbStateNotifyMask, XkbNewKeyboardNotifyMask | XkbMapNotifyMask | XkbStateNotifyMask)) {
+        _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to set XKB events mask");
+        return GLFW_FALSE;
+    }
+    return GLFW_TRUE;
+}
+
+GLFWbool
+glfw_xkb_update_x11_keyboard_id(_GLFWXKBData *xkb) {
+    xkb->keyboard_device_id = -1;
+    xcb_connection_t* conn = XGetXCBConnection(_glfw.x11.display);
+    if (!conn) {
+        _glfwInputError(GLFW_PLATFORM_ERROR, "X11: Failed to retrieve XCB connection");
+        return GLFW_FALSE;
+    }
+
+    xkb->keyboard_device_id = xkb_x11_get_core_keyboard_device_id(conn);
+    if (xkb->keyboard_device_id == -1) {
+        _glfwInputError(GLFW_PLATFORM_ERROR, "X11: Failed to retrieve core keyboard device id");
+        return GLFW_FALSE;
+    }
+    return GLFW_TRUE;
+}
+
+#define xkb_glfw_load_keymap(keymap, ...) {\
+    xcb_connection_t* conn = XGetXCBConnection(_glfw.x11.display); \
+    if (conn) keymap = xkb_x11_keymap_new_from_device(xkb->context, conn, xkb->keyboard_device_id, XKB_KEYMAP_COMPILE_NO_FLAGS); \
+}
+
+#define xkb_glfw_load_state(keymap, state, ...) {\
+    xcb_connection_t* conn = XGetXCBConnection(_glfw.x11.display); \
+    if (conn) state = xkb_x11_state_new_from_device(keymap, conn, xkb->keyboard_device_id); \
+}
+
+#else
+
+#define xkb_glfw_load_keymap(keymap, map_str) keymap = xkb_keymap_new_from_string(xkb->context, map_str, XKB_KEYMAP_FORMAT_TEXT_V1, 0);
+#define xkb_glfw_load_state(keymap, state, ...) state = xkb_state_new(keymap);
+
+#endif
+
+void
+glfw_xkb_release(_GLFWXKBData *xkb) {
+    if (xkb->composeState) {
+        xkb_compose_state_unref(xkb->composeState);
+        xkb->composeState = NULL;
+    }
+    if (xkb->keymap) {
+        xkb_keymap_unref(xkb->keymap);
+        xkb->keymap = NULL;
+    }
+    if (xkb->state) {
+        xkb_state_unref(xkb->state);
+        xkb->state = NULL;
+    }
+    if (xkb->clean_state) {
+        xkb_state_unref(xkb->clean_state);
+        xkb->clean_state = NULL;
+    }
+    if (xkb->context) {
+        xkb_context_unref(xkb->context);
+        xkb->context = NULL;
+    }
+}
+
+GLFWbool
+glfw_xkb_create_context(_GLFWXKBData *xkb) {
+    xkb->context = xkb_context_new(0);
+    debug_keyboard = getenv("GLFW_DEBUG_KEYBOARD") != NULL;
+    if (!xkb->context)
+    {
+        _glfwInputError(GLFW_PLATFORM_ERROR,
+                        "Failed to initialize XKB context");
+        return GLFW_FALSE;
+    }
+    return GLFW_TRUE;
+}
+
+GLFWbool
+glfw_xkb_compile_keymap(_GLFWXKBData *xkb, const char *map_str) {
+    const char* locale = NULL;
+    struct xkb_state* state = NULL, *clean_state = NULL;
+    struct xkb_keymap* keymap = NULL;
+    struct xkb_compose_table* compose_table = NULL;
+    struct xkb_compose_state* compose_state = NULL;
+    (void)(map_str);  // not needed on X11
+    GLFWbool ok = GLFW_FALSE;
+
+    xkb_glfw_load_keymap(keymap, map_str);
+    if (!keymap) _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to compile XKB keymap");
+    else {
+        xkb_glfw_load_state(keymap, state);
+        clean_state = xkb_state_new(keymap);
+        if (!state || !clean_state) {
+            _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to create XKB state");
+            xkb_keymap_unref(keymap); keymap = NULL;
+        } else {
+            ok = GLFW_TRUE;
+            /* Look up the preferred locale, falling back to "C" as default. */
+            locale = getenv("LC_ALL");
+            if (!locale) locale = getenv("LC_CTYPE");
+            if (!locale) locale = getenv("LANG");
+            if (!locale) locale = "C";
+            compose_table = xkb_compose_table_new_from_locale(xkb->context, locale, XKB_COMPOSE_COMPILE_NO_FLAGS);
+            if (!compose_table) {
+                _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to create XKB compose table");
+            } else {
+                compose_state = xkb_compose_state_new(compose_table, XKB_COMPOSE_STATE_NO_FLAGS);
+                xkb_compose_table_unref(compose_table); compose_table = NULL;
+                if (!compose_state) {
+                    _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to create XKB compose state");
+                }
+            }
+        }
+    }
+    if (keymap && state && clean_state) {
+        if (xkb->composeState) xkb_compose_state_unref(xkb->composeState);
+        xkb->composeState = compose_state;
+        if (xkb->keymap) xkb_keymap_unref(xkb->keymap);
+        xkb->keymap = keymap;
+        if (xkb->state) xkb_state_unref(xkb->state);
+        xkb->state = state;
+        if (xkb->clean_state) xkb_state_unref(xkb->clean_state);
+        xkb->clean_state = clean_state;
+    }
+    if (xkb->keymap) {
+#define S(a, n) xkb->a##Idx = xkb_keymap_mod_get_index(xkb->keymap, n); xkb->a##Mask = 1 << xkb->a##Idx;
+        S(control, XKB_MOD_NAME_CTRL);
+        S(alt, XKB_MOD_NAME_ALT);
+        S(shift, XKB_MOD_NAME_SHIFT);
+        S(super, XKB_MOD_NAME_LOGO);
+        S(capsLock, XKB_MOD_NAME_CAPS);
+        S(numLock, XKB_MOD_NAME_NUM);
+#undef S
+        size_t capacity = sizeof(xkb->unknownModifiers)/sizeof(xkb->unknownModifiers[0]), j = 0;
+        for (xkb_mod_index_t i = 0; i < capacity; i++) xkb->unknownModifiers[i] = XKB_MOD_INVALID;
+        for (xkb_mod_index_t i = 0; i < xkb_keymap_num_mods(xkb->keymap) && j < capacity - 1; i++) {
+            if (i != xkb->controlIdx && i != xkb->altIdx && i != xkb->shiftIdx && i != xkb->superIdx && i != xkb->capsLockIdx && i != xkb->numLockIdx) xkb->unknownModifiers[j++] = i;
+        }
+        xkb->modifiers = 0;
+        xkb->activeUnknownModifiers = 0;
+    }
+    return ok;
+}
+
+static inline xkb_mod_mask_t
+active_unknown_modifiers(_GLFWXKBData *xkb) {
+    size_t i = 0;
+    xkb_mod_mask_t ans = 0;
+    while (xkb->unknownModifiers[i] != XKB_MOD_INVALID) {
+        if (xkb_state_mod_index_is_active(xkb->state, xkb->unknownModifiers[i], XKB_STATE_MODS_EFFECTIVE)) ans |= (1 << xkb->unknownModifiers[i]);
+        i++;
+    }
+    return ans;
+}
+
+
+void
+glfw_xkb_update_modifiers(_GLFWXKBData *xkb, xkb_mod_mask_t depressed, xkb_mod_mask_t latched, xkb_mod_mask_t locked, xkb_layout_index_t base_group, xkb_layout_index_t latched_group, xkb_layout_index_t locked_group) {
+    if (!xkb->keymap) return;
+    xkb->modifiers = 0;
+    xkb_state_update_mask(xkb->state, depressed, latched, locked, base_group, latched_group, locked_group);
+    // We have to update the groups in clean_state, as they change for
+    // different keyboard layouts, see https://github.com/kovidgoyal/kitty/issues/488
+    xkb_state_update_mask(xkb->clean_state, 0, 0, 0, base_group, latched_group, locked_group);
+#define S(attr, name) if (xkb_state_mod_index_is_active(xkb->state, xkb->attr##Idx, XKB_STATE_MODS_EFFECTIVE)) xkb->modifiers |= GLFW_MOD_##name
+    S(control, CONTROL); S(alt, ALT); S(shift, SHIFT); S(super, SUPER); S(capsLock, CAPS_LOCK); S(numLock, NUM_LOCK);
+#undef S
+    xkb->activeUnknownModifiers = active_unknown_modifiers(xkb);
+}
+
+GLFWbool
+glfw_xkb_should_repeat(_GLFWXKBData *xkb, xkb_keycode_t scancode) {
+#ifdef _GLFW_WAYLAND
+    scancode += 8;
+#endif
+    return xkb_keymap_key_repeats(xkb->keymap, scancode);
+}
+
+
+static char text[256];
+
+static inline xkb_keysym_t
+compose_symbol(_GLFWXKBData *xkb, xkb_keysym_t sym) {
+    if (sym == XKB_KEY_NoSymbol || !xkb->composeState) return sym;
+    if (xkb_compose_state_feed(xkb->composeState, sym) != XKB_COMPOSE_FEED_ACCEPTED) return sym;
+    switch (xkb_compose_state_get_status(xkb->composeState)) {
+        case XKB_COMPOSE_COMPOSED:
+            xkb_compose_state_get_utf8(xkb->composeState, text, sizeof(text));
+            return xkb_compose_state_get_one_sym(xkb->composeState);
+        case XKB_COMPOSE_COMPOSING:
+        case XKB_COMPOSE_CANCELLED:
+            return XKB_KEY_NoSymbol;
+        case XKB_COMPOSE_NOTHING:
+        default:
+            return sym;
+    }
+}
+
+
+const char*
+glfw_xkb_keysym_name(xkb_keysym_t sym) {
+    static char name[256];
+    name[0] = 0;
+    xkb_keysym_get_name(sym, name, sizeof(name));
+    return name;
+}
+
+
+static inline const char*
+format_mods(unsigned int mods) {
+    static char buf[128];
+    char *p = buf, *s;
+#define pr(x) p += snprintf(p, sizeof(buf) - (p - buf) - 1, "%s", x)
+    pr("mods: ");
+    s = p;
+    if (mods & GLFW_MOD_CONTROL) pr("ctrl+");
+    if (mods & GLFW_MOD_ALT) pr("alt+");
+    if (mods & GLFW_MOD_SHIFT) pr("shift+");
+    if (mods & GLFW_MOD_SUPER) pr("super+");
+    if (mods & GLFW_MOD_CAPS_LOCK) pr("capslock+");
+    if (mods & GLFW_MOD_NUM_LOCK) pr("numlock+");
+    if (p == s) pr("none");
+    else p--;
+    pr(" ");
+#undef pr
+    return buf;
+}
+
+static inline const char*
+format_xkb_mods(_GLFWXKBData *xkb, const char* name, xkb_mod_mask_t mods) {
+    static char buf[512];
+    char *p = buf, *s;
+#define pr(x) p += snprintf(p, sizeof(buf) - (p - buf) - 1, "%s", x)
+    pr(name); pr(": ");
+    s = p;
+    for (xkb_mod_index_t i = 0; i < xkb_keymap_num_mods(xkb->keymap); i++) {
+        xkb_mod_mask_t m = 1 << i;
+        if (m & mods) { pr(xkb_keymap_mod_get_name(xkb->keymap, i)); pr("+"); }
+    }
+    if (p == s) pr("none");
+    else p--;
+    pr(" ");
+#undef pr
+    return buf;
+}
+
+void
+glfw_xkb_handle_key_event(_GLFWwindow *window, _GLFWXKBData *xkb, xkb_keycode_t scancode, int action) {
+    const xkb_keysym_t *syms, *clean_syms;
+    xkb_keysym_t glfw_sym;
+    xkb_keycode_t code_for_sym = scancode;
+#ifdef _GLFW_WAYLAND
+    code_for_sym += 8;
+#endif
+    debug("scancode: 0x%x release: %d ", scancode, action == GLFW_RELEASE);
+    int num_syms = xkb_state_key_get_syms(xkb->state, code_for_sym, &syms);
+    int num_clean_syms = xkb_state_key_get_syms(xkb->clean_state, code_for_sym, &clean_syms);
+    text[0] = 0;
+    // According to the documentation of xkb_compose_state_feed it does not
+    // support multi-sym events, so we ignore them
+    if (num_syms != 1 || num_clean_syms != 1) {
+        debug("scancode: 0x%x num_syms: %d num_clean_syms: %d ignoring event\n", scancode, num_syms, num_clean_syms);
+        return;
+    }
+    glfw_sym = clean_syms[0];
+    debug("clean_sym: %s ", glfw_xkb_keysym_name(clean_syms[0]));
+    if (action == GLFW_PRESS || action == GLFW_REPEAT) {
+        const char *text_type = "composed_text";
+        glfw_sym = compose_symbol(xkb, syms[0]);
+        if (glfw_sym == XKB_KEY_NoSymbol) {
+            debug("compose not complete, ignoring.\n");
+            return;
+        }
+        debug("composed_sym: %s ", glfw_xkb_keysym_name(glfw_sym));
+        if (glfw_sym == syms[0]) { // composed sym is the same as non-composed sym
+            // Only use the clean_sym if no mods other than the mods we report
+            // are active (for example if ISO_Shift_Level_* mods are active
+            // they are not reported by GLFW so the key should be the shifted
+            // key). See https://github.com/kovidgoyal/kitty/issues/171#issuecomment-377557053
+            xkb_mod_mask_t consumed_unknown_mods = xkb_state_key_get_consumed_mods(xkb->state, code_for_sym) & xkb->activeUnknownModifiers;
+            if (xkb->activeUnknownModifiers) debug("%s", format_xkb_mods(xkb, "active_unknown_mods", xkb->activeUnknownModifiers));
+            if (consumed_unknown_mods) { debug("%s", format_xkb_mods(xkb, "consumed_unknown_mods", consumed_unknown_mods)); }
+            else glfw_sym = clean_syms[0];
+            // xkb returns text even if alt and/or super are pressed
+            if ( ((GLFW_MOD_CONTROL | GLFW_MOD_ALT | GLFW_MOD_SUPER) & xkb->modifiers) == 0) xkb_state_key_get_utf8(xkb->state, code_for_sym, text, sizeof(text));
+            text_type = "text";
+        }
+        if ((1 <= text[0] && text[0] <= 31) || text[0] == 127) text[0] = 0;  // dont send text for ascii control codes
+        if (text[0]) { debug("%s: %s ", text_type, text); }
+    }
+    int glfw_keycode = glfw_key_for_sym(glfw_sym);
+    debug("%sglfw_key: %s\n", format_mods(xkb->modifiers), _glfwGetKeyName(glfw_keycode));
+    _glfwInputKeyboard(window, glfw_keycode, glfw_sym, action, xkb->modifiers, text, 0);
+}

glfw/xkb_glfw.h

@@ -1,7 +1,7 @@
 //========================================================================
 // GLFW 3.3 XKB - www.glfw.org
 //------------------------------------------------------------------------
-// Copyright (c) 2014 Kovid Goyal <kovid@kovidgoyal.net>
+// Copyright (c) 2018 Kovid Goyal <kovid@kovidgoyal.net>
 //
 // This software is provided 'as-is', without any express or implied
 // warranty. In no event will the authors be held liable for any damages
@@ -28,347 +28,56 @@
 
 #include <xkbcommon/xkbcommon.h>
 #include <xkbcommon/xkbcommon-compose.h>
-
-typedef struct xkb_context* (* PFN_xkb_context_new)(enum xkb_context_flags);
-typedef void (* PFN_xkb_context_unref)(struct xkb_context*);
-typedef struct xkb_keymap* (* PFN_xkb_keymap_new_from_string)(struct xkb_context*, const char*, enum xkb_keymap_format, enum xkb_keymap_compile_flags);
-typedef void (* PFN_xkb_keymap_unref)(struct xkb_keymap*);
-typedef xkb_mod_index_t (* PFN_xkb_keymap_mod_get_index)(struct xkb_keymap*, const char*);
-typedef int (* PFN_xkb_keymap_key_repeats)(struct xkb_keymap*, xkb_keycode_t);
-typedef struct xkb_state* (* PFN_xkb_state_new)(struct xkb_keymap*);
-typedef void (* PFN_xkb_state_unref)(struct xkb_state*);
-typedef int (* PFN_xkb_state_key_get_syms)(struct xkb_state*, xkb_keycode_t, const xkb_keysym_t**);
-typedef enum xkb_state_component (* PFN_xkb_state_update_mask)(struct xkb_state*, xkb_mod_mask_t, xkb_mod_mask_t, xkb_mod_mask_t, xkb_layout_index_t, xkb_layout_index_t, xkb_layout_index_t);
-typedef xkb_mod_mask_t (* PFN_xkb_state_serialize_mods)(struct xkb_state*, enum xkb_state_component);
-
-#define xkb_context_new GLFW_XKB_GLOBAL_NAME.context_new
-#define xkb_context_unref GLFW_XKB_GLOBAL_NAME.context_unref
-#define xkb_keymap_new_from_string GLFW_XKB_GLOBAL_NAME.keymap_new_from_string
-#define xkb_keymap_unref GLFW_XKB_GLOBAL_NAME.keymap_unref
-#define xkb_keymap_mod_get_index GLFW_XKB_GLOBAL_NAME.keymap_mod_get_index
-#define xkb_keymap_key_repeats GLFW_XKB_GLOBAL_NAME.keymap_key_repeats
-#define xkb_state_new GLFW_XKB_GLOBAL_NAME.state_new
-#define xkb_state_unref GLFW_XKB_GLOBAL_NAME.state_unref
-#define xkb_state_key_get_syms GLFW_XKB_GLOBAL_NAME.state_key_get_syms
-#define xkb_state_update_mask GLFW_XKB_GLOBAL_NAME.state_update_mask
-#define xkb_state_serialize_mods GLFW_XKB_GLOBAL_NAME.state_serialize_mods
-
-typedef struct xkb_compose_table* (* PFN_xkb_compose_table_new_from_locale)(struct xkb_context*, const char*, enum xkb_compose_compile_flags);
-typedef void (* PFN_xkb_compose_table_unref)(struct xkb_compose_table*);
-typedef struct xkb_compose_state* (* PFN_xkb_compose_state_new)(struct xkb_compose_table*, enum xkb_compose_state_flags);
-typedef void (* PFN_xkb_compose_state_unref)(struct xkb_compose_state*);
-typedef enum xkb_compose_feed_result (* PFN_xkb_compose_state_feed)(struct xkb_compose_state*, xkb_keysym_t);
-typedef enum xkb_compose_status (* PFN_xkb_compose_state_get_status)(struct xkb_compose_state*);
-typedef xkb_keysym_t (* PFN_xkb_compose_state_get_one_sym)(struct xkb_compose_state*);
-
-#define xkb_compose_table_new_from_locale GLFW_XKB_GLOBAL_NAME.compose_table_new_from_locale
-#define xkb_compose_table_unref GLFW_XKB_GLOBAL_NAME.compose_table_unref
-#define xkb_compose_state_new GLFW_XKB_GLOBAL_NAME.compose_state_new
-#define xkb_compose_state_unref GLFW_XKB_GLOBAL_NAME.compose_state_unref
-#define xkb_compose_state_feed GLFW_XKB_GLOBAL_NAME.compose_state_feed
-#define xkb_compose_state_get_status GLFW_XKB_GLOBAL_NAME.compose_state_get_status
-#define xkb_compose_state_get_one_sym GLFW_XKB_GLOBAL_NAME.compose_state_get_one_sym
-
+#ifdef _GLFW_X11
+#include <xkbcommon/xkbcommon-x11.h>
+#endif
 
 typedef struct {
-    void*                   handle;
     struct xkb_context*     context;
     struct xkb_keymap*      keymap;
     struct xkb_state*       state;
+    struct xkb_state*       clean_state;
     struct xkb_compose_state* composeState;
-    short int               keycodes[256];
-    short int               scancodes[GLFW_KEY_LAST + 1];
 
+    xkb_mod_index_t         controlIdx;
+    xkb_mod_index_t         altIdx;
+    xkb_mod_index_t         shiftIdx;
+    xkb_mod_index_t         superIdx;
+    xkb_mod_index_t         capsLockIdx;
+    xkb_mod_index_t         numLockIdx;
     xkb_mod_mask_t          controlMask;
     xkb_mod_mask_t          altMask;
     xkb_mod_mask_t          shiftMask;
     xkb_mod_mask_t          superMask;
     xkb_mod_mask_t          capsLockMask;
     xkb_mod_mask_t          numLockMask;
+    xkb_mod_mask_t          activeUnknownModifiers;
     unsigned int            modifiers;
+    xkb_mod_index_t         unknownModifiers[256];
 
-    PFN_xkb_context_new context_new;
-    PFN_xkb_context_unref context_unref;
-    PFN_xkb_keymap_new_from_string keymap_new_from_string;
-    PFN_xkb_keymap_unref keymap_unref;
-    PFN_xkb_keymap_mod_get_index keymap_mod_get_index;
-    PFN_xkb_keymap_key_repeats keymap_key_repeats;
-    PFN_xkb_state_new state_new;
-    PFN_xkb_state_unref state_unref;
-    PFN_xkb_state_key_get_syms state_key_get_syms;
-    PFN_xkb_state_update_mask state_update_mask;
-    PFN_xkb_state_serialize_mods state_serialize_mods;
+#ifdef _GLFW_X11
+    int32_t                 keyboard_device_id;
+    GLFWbool                available;
+    GLFWbool                detectable;
+    int                     majorOpcode;
+    int                     eventBase;
+    int                     errorBase;
+    int                     major;
+    int                     minor;
+#endif
 
-    PFN_xkb_compose_table_new_from_locale compose_table_new_from_locale;
-    PFN_xkb_compose_table_unref compose_table_unref;
-    PFN_xkb_compose_state_new compose_state_new;
-    PFN_xkb_compose_state_unref compose_state_unref;
-    PFN_xkb_compose_state_feed compose_state_feed;
-    PFN_xkb_compose_state_get_status compose_state_get_status;
-    PFN_xkb_compose_state_get_one_sym compose_state_get_one_sym;
 } _GLFWXKBData;
 
-#define bind_xkb_sym(name) GLFW_XKB_GLOBAL_NAME.name = (PFN_xkb_##name) _glfw_dlsym(GLFW_XKB_GLOBAL_NAME.handle, "xkb_" #name)
-#define load_glfw_xkb() {\
-    GLFW_XKB_GLOBAL_NAME.handle = _glfw_dlopen("libxkbcommon.so.0"); \
-    if (!GLFW_XKB_GLOBAL_NAME.handle) \
-    { \
-        _glfwInputError(GLFW_PLATFORM_ERROR, \
-                        "Failed to open libxkbcommon"); \
-        return GLFW_FALSE; \
-    } \
-    bind_xkb_sym(context_new); \
-    bind_xkb_sym(context_unref); \
-    bind_xkb_sym(keymap_new_from_string); \
-    bind_xkb_sym(keymap_unref); \
-    bind_xkb_sym(keymap_mod_get_index); \
-    bind_xkb_sym(keymap_key_repeats); \
-    bind_xkb_sym(state_new); \
-    bind_xkb_sym(state_unref); \
-    bind_xkb_sym(state_key_get_syms); \
-    bind_xkb_sym(state_update_mask); \
-    bind_xkb_sym(state_serialize_mods); \
-    bind_xkb_sym(compose_table_new_from_locale); \
-    bind_xkb_sym(compose_table_unref); \
-    bind_xkb_sym(compose_state_new); \
-    bind_xkb_sym(compose_state_unref); \
-    bind_xkb_sym(compose_state_feed); \
-    bind_xkb_sym(compose_state_get_status); \
-    bind_xkb_sym(compose_state_get_one_sym); \
-}
+#ifdef _GLFW_X11
+GLFWbool glfw_xkb_set_x11_events_mask(void);
+GLFWbool glfw_xkb_update_x11_keyboard_id(_GLFWXKBData *xkb);
+#endif
 
-#define release_glfw_xkb() {\
-    if (GLFW_XKB_GLOBAL_NAME.composeState) { \
-        xkb_compose_state_unref(GLFW_XKB_GLOBAL_NAME.composeState); \
-        GLFW_XKB_GLOBAL_NAME.composeState = NULL; \
-    } \
-    if (GLFW_XKB_GLOBAL_NAME.keymap) { \
-        xkb_keymap_unref(GLFW_XKB_GLOBAL_NAME.keymap); \
-        GLFW_XKB_GLOBAL_NAME.keymap = NULL; \
-    } \
-    if (GLFW_XKB_GLOBAL_NAME.state) { \
-        xkb_state_unref(GLFW_XKB_GLOBAL_NAME.state); \
-        GLFW_XKB_GLOBAL_NAME.state = NULL; \
-    } \
-    if (GLFW_XKB_GLOBAL_NAME.context) { \
-        xkb_context_unref(GLFW_XKB_GLOBAL_NAME.context); \
-        GLFW_XKB_GLOBAL_NAME.context = NULL; \
-    } \
-    if (GLFW_XKB_GLOBAL_NAME.handle) { \
-        _glfw_dlclose(GLFW_XKB_GLOBAL_NAME.handle); \
-        GLFW_XKB_GLOBAL_NAME.handle = NULL; \
-    } \
-}
-
-#define create_glfw_xkb_context() {\
-    GLFW_XKB_GLOBAL_NAME.context = xkb_context_new(0); \
-    if (!GLFW_XKB_GLOBAL_NAME.context) \
-    { \
-        _glfwInputError(GLFW_PLATFORM_ERROR, \
-                        "Failed to initialize XKB context"); \
-        return GLFW_FALSE; \
-    } \
-    int scancode; \
-    memset(GLFW_XKB_GLOBAL_NAME.keycodes, -1, sizeof(GLFW_XKB_GLOBAL_NAME.keycodes)); \
-    memset(GLFW_XKB_GLOBAL_NAME.scancodes, -1, sizeof(GLFW_XKB_GLOBAL_NAME.scancodes)); \
-\
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_GRAVE]      = GLFW_KEY_GRAVE_ACCENT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_1]          = GLFW_KEY_1; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_2]          = GLFW_KEY_2; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_3]          = GLFW_KEY_3; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_4]          = GLFW_KEY_4; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_5]          = GLFW_KEY_5; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_6]          = GLFW_KEY_6; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_7]          = GLFW_KEY_7; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_8]          = GLFW_KEY_8; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_9]          = GLFW_KEY_9; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_0]          = GLFW_KEY_0; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_SPACE]      = GLFW_KEY_SPACE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_MINUS]      = GLFW_KEY_MINUS; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_EQUAL]      = GLFW_KEY_EQUAL; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_Q]          = GLFW_KEY_Q; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_W]          = GLFW_KEY_W; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_E]          = GLFW_KEY_E; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_R]          = GLFW_KEY_R; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_T]          = GLFW_KEY_T; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_Y]          = GLFW_KEY_Y; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_U]          = GLFW_KEY_U; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_I]          = GLFW_KEY_I; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_O]          = GLFW_KEY_O; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_P]          = GLFW_KEY_P; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_LEFTBRACE]  = GLFW_KEY_LEFT_BRACKET; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_RIGHTBRACE] = GLFW_KEY_RIGHT_BRACKET; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_A]          = GLFW_KEY_A; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_S]          = GLFW_KEY_S; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_D]          = GLFW_KEY_D; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F]          = GLFW_KEY_F; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_G]          = GLFW_KEY_G; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_H]          = GLFW_KEY_H; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_J]          = GLFW_KEY_J; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_K]          = GLFW_KEY_K; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_L]          = GLFW_KEY_L; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_SEMICOLON]  = GLFW_KEY_SEMICOLON; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_APOSTROPHE] = GLFW_KEY_APOSTROPHE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_Z]          = GLFW_KEY_Z; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_X]          = GLFW_KEY_X; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_C]          = GLFW_KEY_C; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_V]          = GLFW_KEY_V; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_B]          = GLFW_KEY_B; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_N]          = GLFW_KEY_N; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_M]          = GLFW_KEY_M; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_COMMA]      = GLFW_KEY_COMMA; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_DOT]        = GLFW_KEY_PERIOD; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_SLASH]      = GLFW_KEY_SLASH; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_BACKSLASH]  = GLFW_KEY_BACKSLASH; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_ESC]        = GLFW_KEY_ESCAPE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_TAB]        = GLFW_KEY_TAB; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_LEFTSHIFT]  = GLFW_KEY_LEFT_SHIFT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_RIGHTSHIFT] = GLFW_KEY_RIGHT_SHIFT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_LEFTCTRL]   = GLFW_KEY_LEFT_CONTROL; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_RIGHTCTRL]  = GLFW_KEY_RIGHT_CONTROL; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_LEFTALT]    = GLFW_KEY_LEFT_ALT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_RIGHTALT]   = GLFW_KEY_RIGHT_ALT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_LEFTMETA]   = GLFW_KEY_LEFT_SUPER; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_RIGHTMETA]  = GLFW_KEY_RIGHT_SUPER; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_MENU]       = GLFW_KEY_MENU; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_NUMLOCK]    = GLFW_KEY_NUM_LOCK; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_CAPSLOCK]   = GLFW_KEY_CAPS_LOCK; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_PRINT]      = GLFW_KEY_PRINT_SCREEN; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_SCROLLLOCK] = GLFW_KEY_SCROLL_LOCK; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_PAUSE]      = GLFW_KEY_PAUSE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_DELETE]     = GLFW_KEY_DELETE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_BACKSPACE]  = GLFW_KEY_BACKSPACE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_ENTER]      = GLFW_KEY_ENTER; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_HOME]       = GLFW_KEY_HOME; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_END]        = GLFW_KEY_END; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_PAGEUP]     = GLFW_KEY_PAGE_UP; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_PAGEDOWN]   = GLFW_KEY_PAGE_DOWN; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_INSERT]     = GLFW_KEY_INSERT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_LEFT]       = GLFW_KEY_LEFT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_RIGHT]      = GLFW_KEY_RIGHT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_DOWN]       = GLFW_KEY_DOWN; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_UP]         = GLFW_KEY_UP; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F1]         = GLFW_KEY_F1; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F2]         = GLFW_KEY_F2; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F3]         = GLFW_KEY_F3; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F4]         = GLFW_KEY_F4; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F5]         = GLFW_KEY_F5; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F6]         = GLFW_KEY_F6; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F7]         = GLFW_KEY_F7; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F8]         = GLFW_KEY_F8; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F9]         = GLFW_KEY_F9; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F10]        = GLFW_KEY_F10; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F11]        = GLFW_KEY_F11; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F12]        = GLFW_KEY_F12; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F13]        = GLFW_KEY_F13; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F14]        = GLFW_KEY_F14; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F15]        = GLFW_KEY_F15; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F16]        = GLFW_KEY_F16; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F17]        = GLFW_KEY_F17; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F18]        = GLFW_KEY_F18; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F19]        = GLFW_KEY_F19; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F20]        = GLFW_KEY_F20; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F21]        = GLFW_KEY_F21; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F22]        = GLFW_KEY_F22; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F23]        = GLFW_KEY_F23; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_F24]        = GLFW_KEY_F24; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPSLASH]    = GLFW_KEY_KP_DIVIDE; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPDOT]      = GLFW_KEY_KP_MULTIPLY; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPMINUS]    = GLFW_KEY_KP_SUBTRACT; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPPLUS]     = GLFW_KEY_KP_ADD; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP0]        = GLFW_KEY_KP_0; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP1]        = GLFW_KEY_KP_1; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP2]        = GLFW_KEY_KP_2; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP3]        = GLFW_KEY_KP_3; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP4]        = GLFW_KEY_KP_4; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP5]        = GLFW_KEY_KP_5; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP6]        = GLFW_KEY_KP_6; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP7]        = GLFW_KEY_KP_7; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP8]        = GLFW_KEY_KP_8; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KP9]        = GLFW_KEY_KP_9; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPCOMMA]    = GLFW_KEY_KP_DECIMAL; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPEQUAL]    = GLFW_KEY_KP_EQUAL; \
-    GLFW_XKB_GLOBAL_NAME.keycodes[KEY_KPENTER]    = GLFW_KEY_KP_ENTER; \
-\
-    for (scancode = 0;  scancode < 256;  scancode++) \
-    { \
-        if (GLFW_XKB_GLOBAL_NAME.keycodes[scancode] > 0) \
-            GLFW_XKB_GLOBAL_NAME.scancodes[GLFW_XKB_GLOBAL_NAME.keycodes[scancode]] = scancode; \
-    } \
-\
-}
-
-
-#define xkb_glfw_compile_keymap(map_str) { \
-    const char* locale = NULL; \
-    struct xkb_state* state = NULL;  \
-    struct xkb_keymap* keymap = NULL; \
-    struct xkb_compose_table* compose_table = NULL;  \
-    struct xkb_compose_state* compose_state = NULL;  \
-\
-    keymap = xkb_keymap_new_from_string(GLFW_XKB_GLOBAL_NAME.context, map_str, XKB_KEYMAP_FORMAT_TEXT_V1, 0); \
-    if (!keymap) _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to compile XKB keymap"); \
-    else { \
-        state = xkb_state_new(keymap); \
-        if (!state) { \
-            _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to create XKB state"); \
-            xkb_keymap_unref(keymap); keymap = NULL; \
-        } else { \
-            /* Look up the preferred locale, falling back to "C" as default. */ \
-            locale = getenv("LC_ALL"); \
-            if (!locale) locale = getenv("LC_CTYPE"); \
-            if (!locale) locale = getenv("LANG"); \
-            if (!locale) locale = "C"; \
-            compose_table = xkb_compose_table_new_from_locale(GLFW_XKB_GLOBAL_NAME.context, locale, XKB_COMPOSE_COMPILE_NO_FLAGS); \
-            if (!compose_table) { \
-                _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to create XKB compose table"); \
-                xkb_keymap_unref(keymap); keymap = NULL; \
-                xkb_state_unref(state); state = NULL; \
-            } else { \
-                compose_state = xkb_compose_state_new(compose_table, XKB_COMPOSE_STATE_NO_FLAGS); \
-                xkb_compose_table_unref(compose_table); compose_table = NULL; \
-                if (!compose_state) { \
-                    _glfwInputError(GLFW_PLATFORM_ERROR, "Failed to create XKB compose state"); \
-                    xkb_keymap_unref(keymap); keymap = NULL; \
-                    xkb_state_unref(state); state = NULL; \
-                }\
-            } \
-        } \
-    } \
-    if (keymap && state && compose_state) { \
-        if (GLFW_XKB_GLOBAL_NAME.composeState) xkb_compose_state_unref(GLFW_XKB_GLOBAL_NAME.composeState); \
-        GLFW_XKB_GLOBAL_NAME.composeState = compose_state; \
-        if (GLFW_XKB_GLOBAL_NAME.keymap) xkb_keymap_unref(GLFW_XKB_GLOBAL_NAME.keymap); \
-        GLFW_XKB_GLOBAL_NAME.keymap = keymap; \
-        if (GLFW_XKB_GLOBAL_NAME.state) xkb_state_unref(GLFW_XKB_GLOBAL_NAME.state); \
-        GLFW_XKB_GLOBAL_NAME.state = state; \
-    }\
-    if (GLFW_XKB_GLOBAL_NAME.keymap) { \
-        GLFW_XKB_GLOBAL_NAME.controlMask = 1 << xkb_keymap_mod_get_index(GLFW_XKB_GLOBAL_NAME.keymap, "Control"); \
-        GLFW_XKB_GLOBAL_NAME.altMask = 1 << xkb_keymap_mod_get_index(GLFW_XKB_GLOBAL_NAME.keymap, "Mod1"); \
-        GLFW_XKB_GLOBAL_NAME.shiftMask = 1 << xkb_keymap_mod_get_index(GLFW_XKB_GLOBAL_NAME.keymap, "Shift"); \
-        GLFW_XKB_GLOBAL_NAME.superMask = 1 << xkb_keymap_mod_get_index(GLFW_XKB_GLOBAL_NAME.keymap, "Mod4"); \
-        GLFW_XKB_GLOBAL_NAME.capsLockMask = 1 << xkb_keymap_mod_get_index(GLFW_XKB_GLOBAL_NAME.keymap, "Lock"); \
-        GLFW_XKB_GLOBAL_NAME.numLockMask = 1 << xkb_keymap_mod_get_index(GLFW_XKB_GLOBAL_NAME.keymap, "Mod2"); \
-    } \
-}
-
-
-#define xkb_glfw_update_modifiers(depressed, latched, locked, group) {\
-    xkb_mod_mask_t mask; \
-    unsigned int modifiers = 0; \
-    if (!GLFW_XKB_GLOBAL_NAME.keymap) return; \
-    xkb_state_update_mask(GLFW_XKB_GLOBAL_NAME.state, depressed, latched, locked, 0, 0, group); \
-    mask = xkb_state_serialize_mods(GLFW_XKB_GLOBAL_NAME.state, XKB_STATE_MODS_DEPRESSED | XKB_STATE_LAYOUT_DEPRESSED | XKB_STATE_MODS_LATCHED | XKB_STATE_LAYOUT_LATCHED); \
-    if (mask & GLFW_XKB_GLOBAL_NAME.controlMask) modifiers |= GLFW_MOD_CONTROL; \
-    if (mask & GLFW_XKB_GLOBAL_NAME.altMask) modifiers |= GLFW_MOD_ALT; \
-    if (mask & GLFW_XKB_GLOBAL_NAME.shiftMask) modifiers |= GLFW_MOD_SHIFT; \
-    if (mask & GLFW_XKB_GLOBAL_NAME.superMask) modifiers |= GLFW_MOD_SUPER; \
-    if (mask & GLFW_XKB_GLOBAL_NAME.capsLockMask) modifiers |= GLFW_MOD_CAPS_LOCK; \
-    if (mask & GLFW_XKB_GLOBAL_NAME.numLockMask) modifiers |= GLFW_MOD_NUM_LOCK; \
-    GLFW_XKB_GLOBAL_NAME.modifiers = modifiers; \
-}
-
-
-#define xkb_glfw_to_glfw_key_code(key) \
-    ((key < sizeof(GLFW_XKB_GLOBAL_NAME.keycodes) / sizeof(GLFW_XKB_GLOBAL_NAME.keycodes[0])) ? GLFW_XKB_GLOBAL_NAME.keycodes[key] : GLFW_KEY_UNKNOWN)
+void glfw_xkb_release(_GLFWXKBData *xkb);
+GLFWbool glfw_xkb_create_context(_GLFWXKBData *xkb);
+GLFWbool glfw_xkb_compile_keymap(_GLFWXKBData *xkb, const char *map_str);
+void glfw_xkb_update_modifiers(_GLFWXKBData *xkb, xkb_mod_mask_t depressed, xkb_mod_mask_t latched, xkb_mod_mask_t locked, xkb_layout_index_t base_group, xkb_layout_index_t latched_group, xkb_layout_index_t locked_group);
+GLFWbool glfw_xkb_should_repeat(_GLFWXKBData *xkb, xkb_keycode_t scancode);
+const char* glfw_xkb_keysym_name(xkb_keysym_t sym);
+xkb_keysym_t glfw_xkb_sym_for_key(int key);
+void glfw_xkb_handle_key_event(_GLFWwindow *window, _GLFWXKBData *xkb, xkb_keycode_t scancode, int action);

glfw/xkb_unicode.c

@@ -1,940 +0,0 @@
-//========================================================================
-// GLFW 3.3 X11 - www.glfw.org
-//------------------------------------------------------------------------
-// Copyright (c) 2002-2006 Marcus Geelnard
-// Copyright (c) 2006-2016 Camilla Löwy <elmindreda@glfw.org>
-//
-// This software is provided 'as-is', without any express or implied
-// warranty. In no event will the authors be held liable for any damages
-// arising from the use of this software.
-//
-// Permission is granted to anyone to use this software for any purpose,
-// including commercial applications, and to alter it and redistribute it
-// freely, subject to the following restrictions:
-//
-// 1. The origin of this software must not be misrepresented; you must not
-//    claim that you wrote the original software. If you use this software
-//    in a product, an acknowledgment in the product documentation would
-//    be appreciated but is not required.
-//
-// 2. Altered source versions must be plainly marked as such, and must not
-//    be misrepresented as being the original software.
-//
-// 3. This notice may not be removed or altered from any source
-//    distribution.
-//
-//========================================================================
-
-#include "internal.h"
-
-
-/*
- * Marcus: This code was originally written by Markus G. Kuhn.
- * I have made some slight changes (trimmed it down a bit from >60 KB to
- * 20 KB), but the functionality is the same.
- */
-
-/*
- * This module converts keysym values into the corresponding ISO 10646
- * (UCS, Unicode) values.
- *
- * The array keysymtab[] contains pairs of X11 keysym values for graphical
- * characters and the corresponding Unicode value. The function
- * _glfwKeySym2Unicode() maps a keysym onto a Unicode value using a binary
- * search, therefore keysymtab[] must remain SORTED by keysym value.
- *
- * We allow to represent any UCS character in the range U-00000000 to
- * U-00FFFFFF by a keysym value in the range 0x01000000 to 0x01ffffff.
- * This admittedly does not cover the entire 31-bit space of UCS, but
- * it does cover all of the characters up to U-10FFFF, which can be
- * represented by UTF-16, and more, and it is very unlikely that higher
- * UCS codes will ever be assigned by ISO. So to get Unicode character
- * U+ABCD you can directly use keysym 0x0100abcd.
- *
- * Original author: Markus G. Kuhn <mkuhn@acm.org>, University of
- *                  Cambridge, April 2001
- *
- * Special thanks to Richard Verhoeven <river@win.tue.nl> for preparing
- * an initial draft of the mapping table.
- *
- */
-
-
-//************************************************************************
-//****                KeySym to Unicode mapping table                 ****
-//************************************************************************
-
-static const struct codepair {
-  unsigned short keysym;
-  unsigned short ucs;
-} keysymtab[] = {
-  { 0x01a1, 0x0104 },
-  { 0x01a2, 0x02d8 },
-  { 0x01a3, 0x0141 },
-  { 0x01a5, 0x013d },
-  { 0x01a6, 0x015a },
-  { 0x01a9, 0x0160 },
-  { 0x01aa, 0x015e },
-  { 0x01ab, 0x0164 },
-  { 0x01ac, 0x0179 },
-  { 0x01ae, 0x017d },
-  { 0x01af, 0x017b },
-  { 0x01b1, 0x0105 },
-  { 0x01b2, 0x02db },
-  { 0x01b3, 0x0142 },
-  { 0x01b5, 0x013e },
-  { 0x01b6, 0x015b },
-  { 0x01b7, 0x02c7 },
-  { 0x01b9, 0x0161 },
-  { 0x01ba, 0x015f },
-  { 0x01bb, 0x0165 },
-  { 0x01bc, 0x017a },
-  { 0x01bd, 0x02dd },
-  { 0x01be, 0x017e },
-  { 0x01bf, 0x017c },
-  { 0x01c0, 0x0154 },
-  { 0x01c3, 0x0102 },
-  { 0x01c5, 0x0139 },
-  { 0x01c6, 0x0106 },
-  { 0x01c8, 0x010c },
-  { 0x01ca, 0x0118 },
-  { 0x01cc, 0x011a },
-  { 0x01cf, 0x010e },
-  { 0x01d0, 0x0110 },
-  { 0x01d1, 0x0143 },
-  { 0x01d2, 0x0147 },
-  { 0x01d5, 0x0150 },
-  { 0x01d8, 0x0158 },
-  { 0x01d9, 0x016e },
-  { 0x01db, 0x0170 },
-  { 0x01de, 0x0162 },
-  { 0x01e0, 0x0155 },
-  { 0x01e3, 0x0103 },
-  { 0x01e5, 0x013a },
-  { 0x01e6, 0x0107 },
-  { 0x01e8, 0x010d },
-  { 0x01ea, 0x0119 },
-  { 0x01ec, 0x011b },
-  { 0x01ef, 0x010f },
-  { 0x01f0, 0x0111 },
-  { 0x01f1, 0x0144 },
-  { 0x01f2, 0x0148 },
-  { 0x01f5, 0x0151 },
-  { 0x01f8, 0x0159 },
-  { 0x01f9, 0x016f },
-  { 0x01fb, 0x0171 },
-  { 0x01fe, 0x0163 },
-  { 0x01ff, 0x02d9 },
-  { 0x02a1, 0x0126 },
-  { 0x02a6, 0x0124 },
-  { 0x02a9, 0x0130 },
-  { 0x02ab, 0x011e },
-  { 0x02ac, 0x0134 },
-  { 0x02b1, 0x0127 },
-  { 0x02b6, 0x0125 },
-  { 0x02b9, 0x0131 },
-  { 0x02bb, 0x011f },
-  { 0x02bc, 0x0135 },
-  { 0x02c5, 0x010a },
-  { 0x02c6, 0x0108 },
-  { 0x02d5, 0x0120 },
-  { 0x02d8, 0x011c },
-  { 0x02dd, 0x016c },
-  { 0x02de, 0x015c },
-  { 0x02e5, 0x010b },
-  { 0x02e6, 0x0109 },
-  { 0x02f5, 0x0121 },
-  { 0x02f8, 0x011d },
-  { 0x02fd, 0x016d },
-  { 0x02fe, 0x015d },
-  { 0x03a2, 0x0138 },
-  { 0x03a3, 0x0156 },
-  { 0x03a5, 0x0128 },
-  { 0x03a6, 0x013b },
-  { 0x03aa, 0x0112 },
-  { 0x03ab, 0x0122 },
-  { 0x03ac, 0x0166 },
-  { 0x03b3, 0x0157 },
-  { 0x03b5, 0x0129 },
-  { 0x03b6, 0x013c },
-  { 0x03ba, 0x0113 },
-  { 0x03bb, 0x0123 },
-  { 0x03bc, 0x0167 },
-  { 0x03bd, 0x014a },
-  { 0x03bf, 0x014b },
-  { 0x03c0, 0x0100 },
-  { 0x03c7, 0x012e },
-  { 0x03cc, 0x0116 },
-  { 0x03cf, 0x012a },
-  { 0x03d1, 0x0145 },
-  { 0x03d2, 0x014c },
-  { 0x03d3, 0x0136 },
-  { 0x03d9, 0x0172 },
-  { 0x03dd, 0x0168 },
-  { 0x03de, 0x016a },
-  { 0x03e0, 0x0101 },
-  { 0x03e7, 0x012f },
-  { 0x03ec, 0x0117 },
-  { 0x03ef, 0x012b },
-  { 0x03f1, 0x0146 },
-  { 0x03f2, 0x014d },
-  { 0x03f3, 0x0137 },
-  { 0x03f9, 0x0173 },
-  { 0x03fd, 0x0169 },
-  { 0x03fe, 0x016b },
-  { 0x047e, 0x203e },
-  { 0x04a1, 0x3002 },
-  { 0x04a2, 0x300c },
-  { 0x04a3, 0x300d },
-  { 0x04a4, 0x3001 },
-  { 0x04a5, 0x30fb },
-  { 0x04a6, 0x30f2 },
-  { 0x04a7, 0x30a1 },
-  { 0x04a8, 0x30a3 },
-  { 0x04a9, 0x30a5 },
-  { 0x04aa, 0x30a7 },
-  { 0x04ab, 0x30a9 },
-  { 0x04ac, 0x30e3 },
-  { 0x04ad, 0x30e5 },
-  { 0x04ae, 0x30e7 },
-  { 0x04af, 0x30c3 },
-  { 0x04b0, 0x30fc },
-  { 0x04b1, 0x30a2 },
-  { 0x04b2, 0x30a4 },
-  { 0x04b3, 0x30a6 },
-  { 0x04b4, 0x30a8 },
-  { 0x04b5, 0x30aa },
-  { 0x04b6, 0x30ab },
-  { 0x04b7, 0x30ad },
-  { 0x04b8, 0x30af },
-  { 0x04b9, 0x30b1 },
-  { 0x04ba, 0x30b3 },
-  { 0x04bb, 0x30b5 },
-  { 0x04bc, 0x30b7 },
-  { 0x04bd, 0x30b9 },
-  { 0x04be, 0x30bb },
-  { 0x04bf, 0x30bd },
-  { 0x04c0, 0x30bf },
-  { 0x04c1, 0x30c1 },
-  { 0x04c2, 0x30c4 },
-  { 0x04c3, 0x30c6 },
-  { 0x04c4, 0x30c8 },
-  { 0x04c5, 0x30ca },
-  { 0x04c6, 0x30cb },
-  { 0x04c7, 0x30cc },
-  { 0x04c8, 0x30cd },
-  { 0x04c9, 0x30ce },
-  { 0x04ca, 0x30cf },
-  { 0x04cb, 0x30d2 },
-  { 0x04cc, 0x30d5 },
-  { 0x04cd, 0x30d8 },
-  { 0x04ce, 0x30db },
-  { 0x04cf, 0x30de },
-  { 0x04d0, 0x30df },
-  { 0x04d1, 0x30e0 },
-  { 0x04d2, 0x30e1 },
-  { 0x04d3, 0x30e2 },
-  { 0x04d4, 0x30e4 },
-  { 0x04d5, 0x30e6 },
-  { 0x04d6, 0x30e8 },
-  { 0x04d7, 0x30e9 },
-  { 0x04d8, 0x30ea },
-  { 0x04d9, 0x30eb },
-  { 0x04da, 0x30ec },
-  { 0x04db, 0x30ed },
-  { 0x04dc, 0x30ef },
-  { 0x04dd, 0x30f3 },
-  { 0x04de, 0x309b },
-  { 0x04df, 0x309c },
-  { 0x05ac, 0x060c },
-  { 0x05bb, 0x061b },
-  { 0x05bf, 0x061f },
-  { 0x05c1, 0x0621 },
-  { 0x05c2, 0x0622 },
-  { 0x05c3, 0x0623 },
-  { 0x05c4, 0x0624 },
-  { 0x05c5, 0x0625 },
-  { 0x05c6, 0x0626 },
-  { 0x05c7, 0x0627 },
-  { 0x05c8, 0x0628 },
-  { 0x05c9, 0x0629 },
-  { 0x05ca, 0x062a },
-  { 0x05cb, 0x062b },
-  { 0x05cc, 0x062c },
-  { 0x05cd, 0x062d },
-  { 0x05ce, 0x062e },
-  { 0x05cf, 0x062f },
-  { 0x05d0, 0x0630 },
-  { 0x05d1, 0x0631 },
-  { 0x05d2, 0x0632 },
-  { 0x05d3, 0x0633 },
-  { 0x05d4, 0x0634 },
-  { 0x05d5, 0x0635 },
-  { 0x05d6, 0x0636 },
-  { 0x05d7, 0x0637 },
-  { 0x05d8, 0x0638 },
-  { 0x05d9, 0x0639 },
-  { 0x05da, 0x063a },
-  { 0x05e0, 0x0640 },
-  { 0x05e1, 0x0641 },
-  { 0x05e2, 0x0642 },
-  { 0x05e3, 0x0643 },
-  { 0x05e4, 0x0644 },
-  { 0x05e5, 0x0645 },
-  { 0x05e6, 0x0646 },
-  { 0x05e7, 0x0647 },
-  { 0x05e8, 0x0648 },
-  { 0x05e9, 0x0649 },
-  { 0x05ea, 0x064a },
-  { 0x05eb, 0x064b },
-  { 0x05ec, 0x064c },
-  { 0x05ed, 0x064d },
-  { 0x05ee, 0x064e },
-  { 0x05ef, 0x064f },
-  { 0x05f0, 0x0650 },
-  { 0x05f1, 0x0651 },
-  { 0x05f2, 0x0652 },
-  { 0x06a1, 0x0452 },
-  { 0x06a2, 0x0453 },
-  { 0x06a3, 0x0451 },
-  { 0x06a4, 0x0454 },
-  { 0x06a5, 0x0455 },
-  { 0x06a6, 0x0456 },
-  { 0x06a7, 0x0457 },
-  { 0x06a8, 0x0458 },
-  { 0x06a9, 0x0459 },
-  { 0x06aa, 0x045a },
-  { 0x06ab, 0x045b },
-  { 0x06ac, 0x045c },
-  { 0x06ae, 0x045e },
-  { 0x06af, 0x045f },
-  { 0x06b0, 0x2116 },
-  { 0x06b1, 0x0402 },
-  { 0x06b2, 0x0403 },
-  { 0x06b3, 0x0401 },
-  { 0x06b4, 0x0404 },
-  { 0x06b5, 0x0405 },
-  { 0x06b6, 0x0406 },
-  { 0x06b7, 0x0407 },
-  { 0x06b8, 0x0408 },
-  { 0x06b9, 0x0409 },
-  { 0x06ba, 0x040a },
-  { 0x06bb, 0x040b },
-  { 0x06bc, 0x040c },
-  { 0x06be, 0x040e },
-  { 0x06bf, 0x040f },
-  { 0x06c0, 0x044e },
-  { 0x06c1, 0x0430 },
-  { 0x06c2, 0x0431 },
-  { 0x06c3, 0x0446 },
-  { 0x06c4, 0x0434 },
-  { 0x06c5, 0x0435 },
-  { 0x06c6, 0x0444 },
-  { 0x06c7, 0x0433 },
-  { 0x06c8, 0x0445 },
-  { 0x06c9, 0x0438 },
-  { 0x06ca, 0x0439 },
-  { 0x06cb, 0x043a },
-  { 0x06cc, 0x043b },
-  { 0x06cd, 0x043c },
-  { 0x06ce, 0x043d },
-  { 0x06cf, 0x043e },
-  { 0x06d0, 0x043f },
-  { 0x06d1, 0x044f },
-  { 0x06d2, 0x0440 },
-  { 0x06d3, 0x0441 },
-  { 0x06d4, 0x0442 },
-  { 0x06d5, 0x0443 },
-  { 0x06d6, 0x0436 },
-  { 0x06d7, 0x0432 },
-  { 0x06d8, 0x044c },
-  { 0x06d9, 0x044b },
-  { 0x06da, 0x0437 },
-  { 0x06db, 0x0448 },
-  { 0x06dc, 0x044d },
-  { 0x06dd, 0x0449 },
-  { 0x06de, 0x0447 },
-  { 0x06df, 0x044a },
-  { 0x06e0, 0x042e },
-  { 0x06e1, 0x0410 },
-  { 0x06e2, 0x0411 },
-  { 0x06e3, 0x0426 },
-  { 0x06e4, 0x0414 },
-  { 0x06e5, 0x0415 },
-  { 0x06e6, 0x0424 },
-  { 0x06e7, 0x0413 },
-  { 0x06e8, 0x0425 },
-  { 0x06e9, 0x0418 },
-  { 0x06ea, 0x0419 },
-  { 0x06eb, 0x041a },
-  { 0x06ec, 0x041b },
-  { 0x06ed, 0x041c },
-  { 0x06ee, 0x041d },
-  { 0x06ef, 0x041e },
-  { 0x06f0, 0x041f },
-  { 0x06f1, 0x042f },
-  { 0x06f2, 0x0420 },
-  { 0x06f3, 0x0421 },
-  { 0x06f4, 0x0422 },
-  { 0x06f5, 0x0423 },
-  { 0x06f6, 0x0416 },
-  { 0x06f7, 0x0412 },
-  { 0x06f8, 0x042c },
-  { 0x06f9, 0x042b },
-  { 0x06fa, 0x0417 },
-  { 0x06fb, 0x0428 },
-  { 0x06fc, 0x042d },
-  { 0x06fd, 0x0429 },
-  { 0x06fe, 0x0427 },
-  { 0x06ff, 0x042a },
-  { 0x07a1, 0x0386 },
-  { 0x07a2, 0x0388 },
-  { 0x07a3, 0x0389 },
-  { 0x07a4, 0x038a },
-  { 0x07a5, 0x03aa },
-  { 0x07a7, 0x038c },
-  { 0x07a8, 0x038e },
-  { 0x07a9, 0x03ab },
-  { 0x07ab, 0x038f },
-  { 0x07ae, 0x0385 },
-  { 0x07af, 0x2015 },
-  { 0x07b1, 0x03ac },
-  { 0x07b2, 0x03ad },
-  { 0x07b3, 0x03ae },
-  { 0x07b4, 0x03af },
-  { 0x07b5, 0x03ca },
-  { 0x07b6, 0x0390 },
-  { 0x07b7, 0x03cc },
-  { 0x07b8, 0x03cd },
-  { 0x07b9, 0x03cb },
-  { 0x07ba, 0x03b0 },
-  { 0x07bb, 0x03ce },
-  { 0x07c1, 0x0391 },
-  { 0x07c2, 0x0392 },
-  { 0x07c3, 0x0393 },
-  { 0x07c4, 0x0394 },
-  { 0x07c5, 0x0395 },
-  { 0x07c6, 0x0396 },
-  { 0x07c7, 0x0397 },
-  { 0x07c8, 0x0398 },
-  { 0x07c9, 0x0399 },
-  { 0x07ca, 0x039a },
-  { 0x07cb, 0x039b },
-  { 0x07cc, 0x039c },
-  { 0x07cd, 0x039d },
-  { 0x07ce, 0x039e },
-  { 0x07cf, 0x039f },
-  { 0x07d0, 0x03a0 },
-  { 0x07d1, 0x03a1 },
-  { 0x07d2, 0x03a3 },
-  { 0x07d4, 0x03a4 },
-  { 0x07d5, 0x03a5 },
-  { 0x07d6, 0x03a6 },
-  { 0x07d7, 0x03a7 },
-  { 0x07d8, 0x03a8 },
-  { 0x07d9, 0x03a9 },
-  { 0x07e1, 0x03b1 },
-  { 0x07e2, 0x03b2 },
-  { 0x07e3, 0x03b3 },
-  { 0x07e4, 0x03b4 },
-  { 0x07e5, 0x03b5 },
-  { 0x07e6, 0x03b6 },
-  { 0x07e7, 0x03b7 },
-  { 0x07e8, 0x03b8 },
-  { 0x07e9, 0x03b9 },
-  { 0x07ea, 0x03ba },
-  { 0x07eb, 0x03bb },
-  { 0x07ec, 0x03bc },
-  { 0x07ed, 0x03bd },
-  { 0x07ee, 0x03be },
-  { 0x07ef, 0x03bf },
-  { 0x07f0, 0x03c0 },
-  { 0x07f1, 0x03c1 },
-  { 0x07f2, 0x03c3 },
-  { 0x07f3, 0x03c2 },
-  { 0x07f4, 0x03c4 },
-  { 0x07f5, 0x03c5 },
-  { 0x07f6, 0x03c6 },
-  { 0x07f7, 0x03c7 },
-  { 0x07f8, 0x03c8 },
-  { 0x07f9, 0x03c9 },
-  { 0x08a1, 0x23b7 },
-  { 0x08a2, 0x250c },
-  { 0x08a3, 0x2500 },
-  { 0x08a4, 0x2320 },
-  { 0x08a5, 0x2321 },
-  { 0x08a6, 0x2502 },
-  { 0x08a7, 0x23a1 },
-  { 0x08a8, 0x23a3 },
-  { 0x08a9, 0x23a4 },
-  { 0x08aa, 0x23a6 },
-  { 0x08ab, 0x239b },
-  { 0x08ac, 0x239d },
-  { 0x08ad, 0x239e },
-  { 0x08ae, 0x23a0 },
-  { 0x08af, 0x23a8 },
-  { 0x08b0, 0x23ac },
-  { 0x08bc, 0x2264 },
-  { 0x08bd, 0x2260 },
-  { 0x08be, 0x2265 },
-  { 0x08bf, 0x222b },
-  { 0x08c0, 0x2234 },
-  { 0x08c1, 0x221d },
-  { 0x08c2, 0x221e },
-  { 0x08c5, 0x2207 },
-  { 0x08c8, 0x223c },
-  { 0x08c9, 0x2243 },
-  { 0x08cd, 0x21d4 },
-  { 0x08ce, 0x21d2 },
-  { 0x08cf, 0x2261 },
-  { 0x08d6, 0x221a },
-  { 0x08da, 0x2282 },
-  { 0x08db, 0x2283 },
-  { 0x08dc, 0x2229 },
-  { 0x08dd, 0x222a },
-  { 0x08de, 0x2227 },
-  { 0x08df, 0x2228 },
-  { 0x08ef, 0x2202 },
-  { 0x08f6, 0x0192 },
-  { 0x08fb, 0x2190 },
-  { 0x08fc, 0x2191 },
-  { 0x08fd, 0x2192 },
-  { 0x08fe, 0x2193 },
-  { 0x09e0, 0x25c6 },
-  { 0x09e1, 0x2592 },
-  { 0x09e2, 0x2409 },
-  { 0x09e3, 0x240c },
-  { 0x09e4, 0x240d },
-  { 0x09e5, 0x240a },
-  { 0x09e8, 0x2424 },
-  { 0x09e9, 0x240b },
-  { 0x09ea, 0x2518 },
-  { 0x09eb, 0x2510 },
-  { 0x09ec, 0x250c },
-  { 0x09ed, 0x2514 },
-  { 0x09ee, 0x253c },
-  { 0x09ef, 0x23ba },
-  { 0x09f0, 0x23bb },
-  { 0x09f1, 0x2500 },
-  { 0x09f2, 0x23bc },
-  { 0x09f3, 0x23bd },
-  { 0x09f4, 0x251c },
-  { 0x09f5, 0x2524 },
-  { 0x09f6, 0x2534 },
-  { 0x09f7, 0x252c },
-  { 0x09f8, 0x2502 },
-  { 0x0aa1, 0x2003 },
-  { 0x0aa2, 0x2002 },
-  { 0x0aa3, 0x2004 },
-  { 0x0aa4, 0x2005 },
-  { 0x0aa5, 0x2007 },
-  { 0x0aa6, 0x2008 },
-  { 0x0aa7, 0x2009 },
-  { 0x0aa8, 0x200a },
-  { 0x0aa9, 0x2014 },
-  { 0x0aaa, 0x2013 },
-  { 0x0aae, 0x2026 },
-  { 0x0aaf, 0x2025 },
-  { 0x0ab0, 0x2153 },
-  { 0x0ab1, 0x2154 },
-  { 0x0ab2, 0x2155 },
-  { 0x0ab3, 0x2156 },
-  { 0x0ab4, 0x2157 },
-  { 0x0ab5, 0x2158 },
-  { 0x0ab6, 0x2159 },
-  { 0x0ab7, 0x215a },
-  { 0x0ab8, 0x2105 },
-  { 0x0abb, 0x2012 },
-  { 0x0abc, 0x2329 },
-  { 0x0abe, 0x232a },
-  { 0x0ac3, 0x215b },
-  { 0x0ac4, 0x215c },
-  { 0x0ac5, 0x215d },
-  { 0x0ac6, 0x215e },
-  { 0x0ac9, 0x2122 },
-  { 0x0aca, 0x2613 },
-  { 0x0acc, 0x25c1 },
-  { 0x0acd, 0x25b7 },
-  { 0x0ace, 0x25cb },
-  { 0x0acf, 0x25af },
-  { 0x0ad0, 0x2018 },
-  { 0x0ad1, 0x2019 },
-  { 0x0ad2, 0x201c },
-  { 0x0ad3, 0x201d },
-  { 0x0ad4, 0x211e },
-  { 0x0ad6, 0x2032 },
-  { 0x0ad7, 0x2033 },
-  { 0x0ad9, 0x271d },
-  { 0x0adb, 0x25ac },
-  { 0x0adc, 0x25c0 },
-  { 0x0add, 0x25b6 },
-  { 0x0ade, 0x25cf },
-  { 0x0adf, 0x25ae },
-  { 0x0ae0, 0x25e6 },
-  { 0x0ae1, 0x25ab },
-  { 0x0ae2, 0x25ad },
-  { 0x0ae3, 0x25b3 },
-  { 0x0ae4, 0x25bd },
-  { 0x0ae5, 0x2606 },
-  { 0x0ae6, 0x2022 },
-  { 0x0ae7, 0x25aa },
-  { 0x0ae8, 0x25b2 },
-  { 0x0ae9, 0x25bc },
-  { 0x0aea, 0x261c },
-  { 0x0aeb, 0x261e },
-  { 0x0aec, 0x2663 },
-  { 0x0aed, 0x2666 },
-  { 0x0aee, 0x2665 },
-  { 0x0af0, 0x2720 },
-  { 0x0af1, 0x2020 },
-  { 0x0af2, 0x2021 },
-  { 0x0af3, 0x2713 },
-  { 0x0af4, 0x2717 },
-  { 0x0af5, 0x266f },
-  { 0x0af6, 0x266d },
-  { 0x0af7, 0x2642 },
-  { 0x0af8, 0x2640 },
-  { 0x0af9, 0x260e },
-  { 0x0afa, 0x2315 },
-  { 0x0afb, 0x2117 },
-  { 0x0afc, 0x2038 },
-  { 0x0afd, 0x201a },
-  { 0x0afe, 0x201e },
-  { 0x0ba3, 0x003c },
-  { 0x0ba6, 0x003e },
-  { 0x0ba8, 0x2228 },
-  { 0x0ba9, 0x2227 },
-  { 0x0bc0, 0x00af },
-  { 0x0bc2, 0x22a5 },
-  { 0x0bc3, 0x2229 },
-  { 0x0bc4, 0x230a },
-  { 0x0bc6, 0x005f },
-  { 0x0bca, 0x2218 },
-  { 0x0bcc, 0x2395 },
-  { 0x0bce, 0x22a4 },
-  { 0x0bcf, 0x25cb },
-  { 0x0bd3, 0x2308 },
-  { 0x0bd6, 0x222a },
-  { 0x0bd8, 0x2283 },
-  { 0x0bda, 0x2282 },
-  { 0x0bdc, 0x22a2 },
-  { 0x0bfc, 0x22a3 },
-  { 0x0cdf, 0x2017 },
-  { 0x0ce0, 0x05d0 },
-  { 0x0ce1, 0x05d1 },
-  { 0x0ce2, 0x05d2 },
-  { 0x0ce3, 0x05d3 },
-  { 0x0ce4, 0x05d4 },
-  { 0x0ce5, 0x05d5 },
-  { 0x0ce6, 0x05d6 },
-  { 0x0ce7, 0x05d7 },
-  { 0x0ce8, 0x05d8 },
-  { 0x0ce9, 0x05d9 },
-  { 0x0cea, 0x05da },
-  { 0x0ceb, 0x05db },
-  { 0x0cec, 0x05dc },
-  { 0x0ced, 0x05dd },
-  { 0x0cee, 0x05de },
-  { 0x0cef, 0x05df },
-  { 0x0cf0, 0x05e0 },
-  { 0x0cf1, 0x05e1 },
-  { 0x0cf2, 0x05e2 },
-  { 0x0cf3, 0x05e3 },
-  { 0x0cf4, 0x05e4 },
-  { 0x0cf5, 0x05e5 },
-  { 0x0cf6, 0x05e6 },
-  { 0x0cf7, 0x05e7 },
-  { 0x0cf8, 0x05e8 },
-  { 0x0cf9, 0x05e9 },
-  { 0x0cfa, 0x05ea },
-  { 0x0da1, 0x0e01 },
-  { 0x0da2, 0x0e02 },
-  { 0x0da3, 0x0e03 },
-  { 0x0da4, 0x0e04 },
-  { 0x0da5, 0x0e05 },
-  { 0x0da6, 0x0e06 },
-  { 0x0da7, 0x0e07 },
-  { 0x0da8, 0x0e08 },
-  { 0x0da9, 0x0e09 },
-  { 0x0daa, 0x0e0a },
-  { 0x0dab, 0x0e0b },
-  { 0x0dac, 0x0e0c },
-  { 0x0dad, 0x0e0d },
-  { 0x0dae, 0x0e0e },
-  { 0x0daf, 0x0e0f },
-  { 0x0db0, 0x0e10 },
-  { 0x0db1, 0x0e11 },
-  { 0x0db2, 0x0e12 },
-  { 0x0db3, 0x0e13 },
-  { 0x0db4, 0x0e14 },
-  { 0x0db5, 0x0e15 },
-  { 0x0db6, 0x0e16 },
-  { 0x0db7, 0x0e17 },
-  { 0x0db8, 0x0e18 },
-  { 0x0db9, 0x0e19 },
-  { 0x0dba, 0x0e1a },
-  { 0x0dbb, 0x0e1b },
-  { 0x0dbc, 0x0e1c },
-  { 0x0dbd, 0x0e1d },
-  { 0x0dbe, 0x0e1e },
-  { 0x0dbf, 0x0e1f },
-  { 0x0dc0, 0x0e20 },
-  { 0x0dc1, 0x0e21 },
-  { 0x0dc2, 0x0e22 },
-  { 0x0dc3, 0x0e23 },
-  { 0x0dc4, 0x0e24 },
-  { 0x0dc5, 0x0e25 },
-  { 0x0dc6, 0x0e26 },
-  { 0x0dc7, 0x0e27 },
-  { 0x0dc8, 0x0e28 },
-  { 0x0dc9, 0x0e29 },
-  { 0x0dca, 0x0e2a },
-  { 0x0dcb, 0x0e2b },
-  { 0x0dcc, 0x0e2c },
-  { 0x0dcd, 0x0e2d },
-  { 0x0dce, 0x0e2e },
-  { 0x0dcf, 0x0e2f },
-  { 0x0dd0, 0x0e30 },
-  { 0x0dd1, 0x0e31 },
-  { 0x0dd2, 0x0e32 },
-  { 0x0dd3, 0x0e33 },
-  { 0x0dd4, 0x0e34 },
-  { 0x0dd5, 0x0e35 },
-  { 0x0dd6, 0x0e36 },
-  { 0x0dd7, 0x0e37 },
-  { 0x0dd8, 0x0e38 },
-  { 0x0dd9, 0x0e39 },
-  { 0x0dda, 0x0e3a },
-  { 0x0ddf, 0x0e3f },
-  { 0x0de0, 0x0e40 },
-  { 0x0de1, 0x0e41 },
-  { 0x0de2, 0x0e42 },
-  { 0x0de3, 0x0e43 },
-  { 0x0de4, 0x0e44 },
-  { 0x0de5, 0x0e45 },
-  { 0x0de6, 0x0e46 },
-  { 0x0de7, 0x0e47 },
-  { 0x0de8, 0x0e48 },
-  { 0x0de9, 0x0e49 },
-  { 0x0dea, 0x0e4a },
-  { 0x0deb, 0x0e4b },
-  { 0x0dec, 0x0e4c },
-  { 0x0ded, 0x0e4d },
-  { 0x0df0, 0x0e50 },
-  { 0x0df1, 0x0e51 },
-  { 0x0df2, 0x0e52 },
-  { 0x0df3, 0x0e53 },
-  { 0x0df4, 0x0e54 },
-  { 0x0df5, 0x0e55 },
-  { 0x0df6, 0x0e56 },
-  { 0x0df7, 0x0e57 },
-  { 0x0df8, 0x0e58 },
-  { 0x0df9, 0x0e59 },
-  { 0x0ea1, 0x3131 },
-  { 0x0ea2, 0x3132 },
-  { 0x0ea3, 0x3133 },
-  { 0x0ea4, 0x3134 },
-  { 0x0ea5, 0x3135 },
-  { 0x0ea6, 0x3136 },
-  { 0x0ea7, 0x3137 },
-  { 0x0ea8, 0x3138 },
-  { 0x0ea9, 0x3139 },
-  { 0x0eaa, 0x313a },
-  { 0x0eab, 0x313b },
-  { 0x0eac, 0x313c },
-  { 0x0ead, 0x313d },
-  { 0x0eae, 0x313e },
-  { 0x0eaf, 0x313f },
-  { 0x0eb0, 0x3140 },
-  { 0x0eb1, 0x3141 },
-  { 0x0eb2, 0x3142 },
-  { 0x0eb3, 0x3143 },
-  { 0x0eb4, 0x3144 },
-  { 0x0eb5, 0x3145 },
-  { 0x0eb6, 0x3146 },
-  { 0x0eb7, 0x3147 },
-  { 0x0eb8, 0x3148 },
-  { 0x0eb9, 0x3149 },
-  { 0x0eba, 0x314a },
-  { 0x0ebb, 0x314b },
-  { 0x0ebc, 0x314c },
-  { 0x0ebd, 0x314d },
-  { 0x0ebe, 0x314e },
-  { 0x0ebf, 0x314f },
-  { 0x0ec0, 0x3150 },
-  { 0x0ec1, 0x3151 },
-  { 0x0ec2, 0x3152 },
-  { 0x0ec3, 0x3153 },
-  { 0x0ec4, 0x3154 },
-  { 0x0ec5, 0x3155 },
-  { 0x0ec6, 0x3156 },
-  { 0x0ec7, 0x3157 },
-  { 0x0ec8, 0x3158 },
-  { 0x0ec9, 0x3159 },
-  { 0x0eca, 0x315a },
-  { 0x0ecb, 0x315b },
-  { 0x0ecc, 0x315c },
-  { 0x0ecd, 0x315d },
-  { 0x0ece, 0x315e },
-  { 0x0ecf, 0x315f },
-  { 0x0ed0, 0x3160 },
-  { 0x0ed1, 0x3161 },
-  { 0x0ed2, 0x3162 },
-  { 0x0ed3, 0x3163 },
-  { 0x0ed4, 0x11a8 },
-  { 0x0ed5, 0x11a9 },
-  { 0x0ed6, 0x11aa },
-  { 0x0ed7, 0x11ab },
-  { 0x0ed8, 0x11ac },
-  { 0x0ed9, 0x11ad },
-  { 0x0eda, 0x11ae },
-  { 0x0edb, 0x11af },
-  { 0x0edc, 0x11b0 },
-  { 0x0edd, 0x11b1 },
-  { 0x0ede, 0x11b2 },
-  { 0x0edf, 0x11b3 },
-  { 0x0ee0, 0x11b4 },
-  { 0x0ee1, 0x11b5 },
-  { 0x0ee2, 0x11b6 },
-  { 0x0ee3, 0x11b7 },
-  { 0x0ee4, 0x11b8 },
-  { 0x0ee5, 0x11b9 },
-  { 0x0ee6, 0x11ba },
-  { 0x0ee7, 0x11bb },
-  { 0x0ee8, 0x11bc },
-  { 0x0ee9, 0x11bd },
-  { 0x0eea, 0x11be },
-  { 0x0eeb, 0x11bf },
-  { 0x0eec, 0x11c0 },
-  { 0x0eed, 0x11c1 },
-  { 0x0eee, 0x11c2 },
-  { 0x0eef, 0x316d },
-  { 0x0ef0, 0x3171 },
-  { 0x0ef1, 0x3178 },
-  { 0x0ef2, 0x317f },
-  { 0x0ef3, 0x3181 },
-  { 0x0ef4, 0x3184 },
-  { 0x0ef5, 0x3186 },
-  { 0x0ef6, 0x318d },
-  { 0x0ef7, 0x318e },
-  { 0x0ef8, 0x11eb },
-  { 0x0ef9, 0x11f0 },
-  { 0x0efa, 0x11f9 },
-  { 0x0eff, 0x20a9 },
-  { 0x13a4, 0x20ac },
-  { 0x13bc, 0x0152 },
-  { 0x13bd, 0x0153 },
-  { 0x13be, 0x0178 },
-  { 0x20ac, 0x20ac },
-  { 0xfe50,    '`' },
-  { 0xfe51, 0x00b4 },
-  { 0xfe52,    '^' },
-  { 0xfe53,    '~' },
-  { 0xfe54, 0x00af },
-  { 0xfe55, 0x02d8 },
-  { 0xfe56, 0x02d9 },
-  { 0xfe57, 0x00a8 },
-  { 0xfe58, 0x02da },
-  { 0xfe59, 0x02dd },
-  { 0xfe5a, 0x02c7 },
-  { 0xfe5b, 0x00b8 },
-  { 0xfe5c, 0x02db },
-  { 0xfe5d, 0x037a },
-  { 0xfe5e, 0x309b },
-  { 0xfe5f, 0x309c },
-  { 0xfe63,    '/' },
-  { 0xfe64, 0x02bc },
-  { 0xfe65, 0x02bd },
-  { 0xfe66, 0x02f5 },
-  { 0xfe67, 0x02f3 },
-  { 0xfe68, 0x02cd },
-  { 0xfe69, 0xa788 },
-  { 0xfe6a, 0x02f7 },
-  { 0xfe6e,    ',' },
-  { 0xfe6f, 0x00a4 },
-  { 0xfe80,    'a' }, // XK_dead_a
-  { 0xfe81,    'A' }, // XK_dead_A
-  { 0xfe82,    'e' }, // XK_dead_e
-  { 0xfe83,    'E' }, // XK_dead_E
-  { 0xfe84,    'i' }, // XK_dead_i
-  { 0xfe85,    'I' }, // XK_dead_I
-  { 0xfe86,    'o' }, // XK_dead_o
-  { 0xfe87,    'O' }, // XK_dead_O
-  { 0xfe88,    'u' }, // XK_dead_u
-  { 0xfe89,    'U' }, // XK_dead_U
-  { 0xfe8a, 0x0259 },
-  { 0xfe8b, 0x018f },
-  { 0xfe8c, 0x00b5 },
-  { 0xfe90,    '_' },
-  { 0xfe91, 0x02c8 },
-  { 0xfe92, 0x02cc },
-  { 0xff80 /*XKB_KEY_KP_Space*/,     ' ' },
-  { 0xff95 /*XKB_KEY_KP_7*/, 0x0037 },
-  { 0xff96 /*XKB_KEY_KP_4*/, 0x0034 },
-  { 0xff97 /*XKB_KEY_KP_8*/, 0x0038 },
-  { 0xff98 /*XKB_KEY_KP_6*/, 0x0036 },
-  { 0xff99 /*XKB_KEY_KP_2*/, 0x0032 },
-  { 0xff9a /*XKB_KEY_KP_9*/, 0x0039 },
-  { 0xff9b /*XKB_KEY_KP_3*/, 0x0033 },
-  { 0xff9c /*XKB_KEY_KP_1*/, 0x0031 },
-  { 0xff9d /*XKB_KEY_KP_5*/, 0x0035 },
-  { 0xff9e /*XKB_KEY_KP_0*/, 0x0030 },
-  { 0xffaa /*XKB_KEY_KP_Multiply*/,  '*' },
-  { 0xffab /*XKB_KEY_KP_Add*/,       '+' },
-  { 0xffac /*XKB_KEY_KP_Separator*/, ',' },
-  { 0xffad /*XKB_KEY_KP_Subtract*/,  '-' },
-  { 0xffae /*XKB_KEY_KP_Decimal*/,   '.' },
-  { 0xffaf /*XKB_KEY_KP_Divide*/,    '/' },
-  { 0xffb0 /*XKB_KEY_KP_0*/, 0x0030 },
-  { 0xffb1 /*XKB_KEY_KP_1*/, 0x0031 },
-  { 0xffb2 /*XKB_KEY_KP_2*/, 0x0032 },
-  { 0xffb3 /*XKB_KEY_KP_3*/, 0x0033 },
-  { 0xffb4 /*XKB_KEY_KP_4*/, 0x0034 },
-  { 0xffb5 /*XKB_KEY_KP_5*/, 0x0035 },
-  { 0xffb6 /*XKB_KEY_KP_6*/, 0x0036 },
-  { 0xffb7 /*XKB_KEY_KP_7*/, 0x0037 },
-  { 0xffb8 /*XKB_KEY_KP_8*/, 0x0038 },
-  { 0xffb9 /*XKB_KEY_KP_9*/, 0x0039 },
-  { 0xffbd /*XKB_KEY_KP_Equal*/,     '=' }
-};
-
-
-//////////////////////////////////////////////////////////////////////////
-//////                       GLFW internal API                      //////
-//////////////////////////////////////////////////////////////////////////
-
-// Convert XKB KeySym to Unicode
-//
-long _glfwKeySym2Unicode(unsigned int keysym)
-{
-    int min = 0;
-    int max = sizeof(keysymtab) / sizeof(struct codepair) - 1;
-    int mid;
-
-    // First check for Latin-1 characters (1:1 mapping)
-    if ((keysym >= 0x0020 && keysym <= 0x007e) ||
-        (keysym >= 0x00a0 && keysym <= 0x00ff))
-    {
-        return keysym;
-    }
-
-    // Also check for directly encoded 24-bit UCS characters
-    if ((keysym & 0xff000000) == 0x01000000)
-        return keysym & 0x00ffffff;
-
-    // Binary search in table
-    while (max >= min)
-    {
-        mid = (min + max) / 2;
-        if (keysymtab[mid].keysym < keysym)
-            min = mid + 1;
-        else if (keysymtab[mid].keysym > keysym)
-            max = mid - 1;
-        else
-            return keysymtab[mid].ucs;
-    }
-
-    // No matching Unicode value found
-    return -1;
-}
-

glfw/xkb_unicode.h

@@ -1,28 +0,0 @@
-//========================================================================
-// GLFW 3.3 Linux - www.glfw.org
-//------------------------------------------------------------------------
-// Copyright (c) 2014 Jonas Ådahl <jadahl@gmail.com>
-//
-// This software is provided 'as-is', without any express or implied
-// warranty. In no event will the authors be held liable for any damages
-// arising from the use of this software.
-//
-// Permission is granted to anyone to use this software for any purpose,
-// including commercial applications, and to alter it and redistribute it
-// freely, subject to the following restrictions:
-//
-// 1. The origin of this software must not be misrepresented; you must not
-//    claim that you wrote the original software. If you use this software
-//    in a product, an acknowledgment in the product documentation would
-//    be appreciated but is not required.
-//
-// 2. Altered source versions must be plainly marked as such, and must not
-//    be misrepresented as being the original software.
-//
-// 3. This notice may not be removed or altered from any source
-//    distribution.
-//
-//========================================================================
-
-long _glfwKeySym2Unicode(unsigned int keysym);
-

kittens/ask/main.py

@@ -2,16 +2,15 @@
 # vim:fileencoding=utf-8
 # License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
 
-import json
 import os
-import readline
-import sys
 
 from kitty.cli import parse_args
 from kitty.constants import cache_dir
 
 from ..tui.operations import alternate_screen, styled
 
+readline = None
+
 
 def get_history_items():
     return list(map(readline.get_history_item, range(1, readline.get_current_history_length() + 1)))
@@ -54,7 +53,6 @@ class HistoryCompleter:
             if os.path.exists(self.history_path):
                 readline.read_history_file(self.history_path)
             readline.set_completer(self.complete)
-            readline.parse_and_bind('tab: complete')
         return self
 
     def __exit__(self, *a):
@@ -81,16 +79,25 @@ be used for completions and via the browse history readline bindings.
 '''
 
 
-def real_main(args):
+def main(args):
+    # For some reason importing readline in a key handler in the main kitty process
+    # causes a crash of the python interpreter, probably because of some global
+    # lock
+    global readline
+    import readline as rl
+    readline = rl
+    from kitty.shell import init_readline
     msg = 'Ask the user for input'
     try:
         args, items = parse_args(args[1:], option_text, '', msg, 'kitty ask')
     except SystemExit as e:
-        print(e.args[0])
-        input('Press enter to quit...')
-        raise SystemExit(1)
+        if e.code != 0:
+            print(e.args[0])
+            input('Press enter to quit...')
+        raise SystemExit(e.code)
 
-    readline.read_init_file()
+    init_readline(readline)
+    ans = {'items': items}
 
     with alternate_screen(), HistoryCompleter(args.name):
         if args.message:
@@ -98,17 +105,20 @@ def real_main(args):
 
         prompt = '> '
         try:
-            ans = input(prompt)
+            ans['response'] = input(prompt)
         except (KeyboardInterrupt, EOFError):
-            return
-    print('OK:', json.dumps(ans))
+            pass
+    return ans
 
 
-def main(args=sys.argv):
-    try:
-        real_main(args)
-    except Exception as e:
-        import traceback
-        traceback.print_exc(file=sys.stdout)
-        input('Press enter to quit...')
-        raise SystemExit(1)
+def handle_result(args, data, target_window_id, boss):
+    if 'response' in data:
+        func, *args = data['items']
+        getattr(boss, func)(data['response'], *args)
+
+
+if __name__ == '__main__':
+    import sys
+    ans = main(sys.argv)
+    if ans:
+        print(ans)

kittens/diff/README.asciidoc

@@ -0,0 +1,105 @@
+= kitty-diff - A side-by-side diff tool with syntax highlighting and images
+:toc:
+:toc-placement!:
+
+
+== Major Features
+
+* Displays diffs side-by-side in the kitty terminal.
+
+* Does syntax highlighting of displayed diffs
+
+* Displays images as well as text diffs, even over SSH
+
+* Does recursive directory diffing
+
+
+image::../../screenshots/diff.png?raw=true[Screenshot, showing a sample diff]
+
+toc::[]
+
+
+== Installation
+
+Simply install link:https://github.com/kovidgoyal/kitty[kitty].  You also need
+to have either the link:https://git-scm.com/[git] program or the `diff` program
+installed. Additionally, for syntax highlighting to work,
+link:http://pygments.org/[pygments] must be installed (note that pygments is
+included in the macOS kitty app).
+
+
+== Usage
+
+In the kitty terminal, run:
+
+....
+kitty +kitten diff file1 file2
+....
+
+to see the diff between file1 and file2.
+
+Create an alias in your shell's startup file to shorten the command, for example:
+
+```sh
+alias d="kitty +kitten diff"
+```
+
+Now all you need to do to diff two files is:
+
+```
+d file1 file2
+```
+
+You can also pass directories instead of files to see the recursive diff of the
+directory contents.
+
+
+== Keyboard controls
+
+|===
+|Action |Shortcut
+
+|Quit             | `q, Ctrl+c`
+|Scroll line up   | `k, up`
+|Scroll line down | `j, down`
+|Scroll page up   | `PgUp`
+|Scroll page down | `PgDn`
+|Scroll to top    | `Home`
+|Scroll to bottom | `End`
+|Scroll to next change | `n`
+|Scroll to previous change | `p`
+
+|Increase lines of context | `+`
+|Decrease lines of context | `-`
+|All lines of context      | `a`
+|Restore context to default| `=`
+
+|===
+
+[options="header"]
+
+
+== Integrating with git
+
+Add the following to `~/.gitconfig`:
+
+```gitconfig
+[diff]
+    tool = kitty
+    guitool = kitty.gui
+[difftool]
+    prompt = false
+    trustExitCode = true
+[difftool "kitty"]
+    cmd = kitty +kitten diff $LOCAL $REMOTE
+[difftool "kitty.gui"]
+    cmd = kitty kitty +kitten diff $LOCAL $REMOTE
+```
+
+Now to use kitty-diff to view git diffs, you can simply do:
+
+```
+git difftool --no-symlinks --dir-diff
+```
+
+Once again, creating an alias for this command is useful.

kittens/diff/collect.py

@@ -0,0 +1,189 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import os
+import re
+from functools import lru_cache
+from hashlib import md5
+from mimetypes import guess_type
+
+path_name_map = {}
+
+
+class Segment:
+
+    __slots__ = ('start', 'end', 'start_code', 'end_code')
+
+    def __init__(self, start, start_code):
+        self.start = start
+        self.start_code = start_code
+
+    def __repr__(self):
+        return 'Segment(start={!r}, start_code={!r}, end={!r}, end_code={!r})'.format(
+                self.start, self.start_code, getattr(self, 'end', None), getattr(self, 'end_code', None)
+        )
+
+
+class Collection:
+
+    def __init__(self):
+        self.changes = {}
+        self.renames = {}
+        self.adds = set()
+        self.removes = set()
+        self.all_paths = []
+        self.type_map = {}
+        self.added_count = self.removed_count = 0
+
+    def add_change(self, left_path, right_path):
+        self.changes[left_path] = right_path
+        self.all_paths.append(left_path)
+        self.type_map[left_path] = 'diff'
+
+    def add_rename(self, left_path, right_path):
+        self.renames[left_path] = right_path
+        self.all_paths.append(left_path)
+        self.type_map[left_path] = 'rename'
+
+    def add_add(self, right_path):
+        self.adds.add(right_path)
+        self.all_paths.append(right_path)
+        self.type_map[right_path] = 'add'
+        if isinstance(data_for_path(right_path), str):
+            self.added_count += len(lines_for_path(right_path))
+
+    def add_removal(self, left_path):
+        self.removes.add(left_path)
+        self.all_paths.append(left_path)
+        self.type_map[left_path] = 'removal'
+        if isinstance(data_for_path(left_path), str):
+            self.removed_count += len(lines_for_path(left_path))
+
+    def finalize(self):
+        self.all_paths.sort(key=path_name_map.get)
+
+    def __iter__(self):
+        for path in self.all_paths:
+            typ = self.type_map[path]
+            if typ == 'diff':
+                data = self.changes[path]
+            elif typ == 'rename':
+                data = self.renames[path]
+            else:
+                data = None
+            yield path, typ, data
+
+    def __len__(self):
+        return len(self.all_paths)
+
+
+def collect_files(collection, left, right):
+    left_names, right_names = set(), set()
+    left_path_map, right_path_map = {}, {}
+
+    def walk(base, names, pmap):
+        for dirpath, dirnames, filenames in os.walk(base):
+            for filename in filenames:
+                path = os.path.abspath(os.path.join(dirpath, filename))
+                path_name_map[path] = name = os.path.relpath(path, base)
+                names.add(name)
+                pmap[name] = path
+
+    walk(left, left_names, left_path_map), walk(right, right_names, right_path_map)
+    common_names = left_names & right_names
+    changed_names = {n for n in common_names if data_for_path(left_path_map[n]) != data_for_path(right_path_map[n])}
+    for n in changed_names:
+        collection.add_change(left_path_map[n], right_path_map[n])
+
+    removed = left_names - common_names
+    added = right_names - common_names
+    ahash = {a: hash_for_path(right_path_map[a]) for a in added}
+    rhash = {r: hash_for_path(left_path_map[r]) for r in removed}
+    for name, rh in rhash.items():
+        for n, ah in ahash.items():
+            if ah == rh and data_for_path(left_path_map[name]) == data_for_path(right_path_map[n]):
+                collection.add_rename(left_path_map[name], right_path_map[n])
+                added.discard(n)
+                break
+        else:
+            collection.add_removal(left_path_map[name])
+
+    for name in added:
+        collection.add_add(right_path_map[name])
+
+
+sanitize_pat = re.compile('[\x00-\x09\x0b-\x1f\x7f\x80-\x9f]')
+
+
+def sanitize_sub(m):
+    return '<{:x}>'.format(ord(m.group()[0]))
+
+
+def sanitize(text):
+    return sanitize_pat.sub(sanitize_sub, text)
+
+
+@lru_cache(maxsize=1024)
+def mime_type_for_path(path):
+    return guess_type(path)[0] or 'application/octet-stream'
+
+
+@lru_cache(maxsize=1024)
+def raw_data_for_path(path):
+    with open(path, 'rb') as f:
+        return f.read()
+
+
+def is_image(path):
+    return mime_type_for_path(path).startswith('image/')
+
+
+@lru_cache(maxsize=1024)
+def data_for_path(path):
+    ans = raw_data_for_path(path)
+    if not is_image(path) and not os.path.samefile(path, os.devnull):
+        try:
+            ans = ans.decode('utf-8')
+        except UnicodeDecodeError:
+            pass
+    return ans
+
+
+@lru_cache(maxsize=1024)
+def lines_for_path(path):
+    data = data_for_path(path).replace('\t', lines_for_path.replace_tab_by)
+    return tuple(sanitize(data).splitlines())
+
+
+lines_for_path.replace_tab_by = ' ' * 4
+
+
+@lru_cache(maxsize=1024)
+def hash_for_path(path):
+    return md5(raw_data_for_path(path)).digest()
+
+
+def create_collection(left, right):
+    collection = Collection()
+    if os.path.isdir(left):
+        collect_files(collection, left, right)
+    else:
+        pl, pr = os.path.abspath(left), os.path.abspath(right)
+        path_name_map[pl] = left
+        path_name_map[pr] = right
+        collection.add_change(pl, pr)
+    collection.finalize()
+    return collection
+
+
+highlight_data = {}
+
+
+def set_highlight_data(data):
+    global highlight_data
+    highlight_data = data
+
+
+def highlights_for_path(path):
+    return highlight_data.get(path, [])

kittens/diff/config.py

@@ -0,0 +1,111 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import os
+
+from kitty.config_utils import (
+    init_config, load_config as _load_config, merge_dicts, parse_config_base,
+    python_string, resolve_config, to_color
+)
+from kitty.constants import config_dir
+from kitty.rgb import color_as_sgr
+
+defaults = None
+default_config_path = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), 'diff.conf'
+)
+
+formats = {
+    'title': '',
+    'margin': '',
+    'text': '',
+}
+
+
+def set_formats(opts):
+    formats['text'] = '48' + color_as_sgr(opts.background)
+    formats['title'] = '38' + color_as_sgr(opts.title_fg) + ';48' + color_as_sgr(opts.title_bg) + ';1'
+    formats['margin'] = '38' + color_as_sgr(opts.margin_fg) + ';48' + color_as_sgr(opts.margin_bg)
+    formats['added_margin'] = '38' + color_as_sgr(opts.margin_fg) + ';48' + color_as_sgr(opts.added_margin_bg)
+    formats['removed_margin'] = '38' + color_as_sgr(opts.margin_fg) + ';48' + color_as_sgr(opts.removed_margin_bg)
+    formats['added'] = '48' + color_as_sgr(opts.added_bg)
+    formats['removed'] = '48' + color_as_sgr(opts.removed_bg)
+    formats['filler'] = '48' + color_as_sgr(opts.filler_bg)
+    formats['hunk_margin'] = '38' + color_as_sgr(opts.margin_fg) + ';48' + color_as_sgr(opts.hunk_margin_bg)
+    formats['hunk'] = '38' + color_as_sgr(opts.margin_fg) + ';48' + color_as_sgr(opts.hunk_bg)
+    formats['removed_highlight'] = '48' + color_as_sgr(opts.highlight_removed_bg)
+    formats['added_highlight'] = '48' + color_as_sgr(opts.highlight_added_bg)
+
+
+def syntax_aliases(raw):
+    ans = {}
+    for x in raw.split():
+        a, b = x.partition(':')[::2]
+        if a and b:
+            ans[a.lower()] = b
+    return ans
+
+
+type_map = {
+    'syntax_aliases': syntax_aliases,
+    'num_context_lines': int,
+    'replace_tab_by': python_string,
+}
+
+for name in (
+    'foreground background title_fg title_bg margin_bg margin_fg removed_bg removed_margin_bg added_bg added_margin_bg filler_bg hunk_bg hunk_margin_bg'
+    ' highlight_removed_bg highlight_added_bg'
+).split():
+    type_map[name] = to_color
+
+
+def special_handling(*a):
+    pass
+
+
+def parse_config(lines, check_keys=True):
+    ans = {}
+    parse_config_base(
+        lines,
+        defaults,
+        type_map,
+        special_handling,
+        ans,
+        check_keys=check_keys
+    )
+    return ans
+
+
+def merge_configs(defaults, vals):
+    ans = {}
+    for k, v in defaults.items():
+        if isinstance(v, dict):
+            newvals = vals.get(k, {})
+            ans[k] = merge_dicts(v, newvals)
+        else:
+            ans[k] = vals.get(k, v)
+    return ans
+
+
+def parse_defaults(lines, check_keys=False):
+    return parse_config(lines, check_keys)
+
+
+Options, defaults = init_config(default_config_path, parse_defaults)
+
+
+def load_config(*paths, overrides=None):
+    return _load_config(Options, defaults, parse_config, merge_configs, *paths, overrides=overrides)
+
+
+SYSTEM_CONF = '/etc/xdg/kitty/diff.conf'
+defconf = os.path.join(config_dir, 'diff.conf')
+
+
+def init_config(args):
+    config = tuple(resolve_config(SYSTEM_CONF, defconf, args.config))
+    overrides = (a.replace('=', ' ', 1) for a in args.override or ())
+    opts = load_config(*config, overrides=overrides)
+    set_formats(opts)
+    return opts

kittens/diff/diff.conf

@@ -0,0 +1,38 @@
+# vim:fileencoding=utf-8:ft=conf
+
+# File extension aliases for syntax highlight
+# For example, to syntax highlight file.xyz as
+# file.abc use a setting of xyz:abc
+syntax_aliases pyj:py recipe:py
+
+# The pygments color scheme to use. See https://help.farbox.com/pygments.html
+# for a list of schemes.
+pygments_style default
+
+# The number of lines of context to show around each change.
+num_context_lines 3
+
+# The diff command to use. Must contain the placeholder _CONTEXT_
+# which will be replaced by the number of lines of context. The default
+# is to search the system for either git or diff and use that, if found.
+diff_cmd auto
+
+# The string to replace tabs with. Default is to use four spaces.
+replace_tab_by \x20\x20\x20\x20
+
+# Colors
+foreground            black
+background            white
+title_fg              black
+title_bg              white
+margin_bg             #fafbfc
+margin_fg             #aaaaaa
+removed_bg            #ffeef0
+highlight_removed_bg  #fdb8c0
+removed_margin_bg     #ffdce0
+added_bg              #e6ffed
+highlight_added_bg    #acf2bd
+added_margin_bg       #cdffd8
+filler_bg             #fafbfc
+hunk_margin_bg        #dbedff
+hunk_bg               #f1f8ff

kittens/diff/highlight.py

@@ -0,0 +1,162 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import concurrent
+import os
+import re
+
+from pygments import highlight
+from pygments.formatter import Formatter
+from pygments.lexers import get_lexer_for_filename
+from pygments.util import ClassNotFound
+
+from kitty.rgb import color_as_sgr, parse_sharp
+
+from .collect import Segment, data_for_path, lines_for_path
+
+
+class StyleNotFound(Exception):
+    pass
+
+
+class DiffFormatter(Formatter):
+
+    def __init__(self, style='default'):
+        try:
+            Formatter.__init__(self, style=style)
+            initialized = True
+        except ClassNotFound:
+            initialized = False
+        if not initialized:
+            raise StyleNotFound('pygments style "{}" not found'.format(style))
+
+        self.styles = {}
+        for token, style in self.style:
+            start = []
+            end = []
+            # a style item is a tuple in the following form:
+            # colors are readily specified in hex: 'RRGGBB'
+            if style['color']:
+                start.append('38' + color_as_sgr(parse_sharp(style['color'])))
+                end.append('39')
+            if style['bold']:
+                start.append('1')
+                end.append('22')
+            if style['italic']:
+                start.append('3')
+                end.append('23')
+            if style['underline']:
+                start.append('4')
+                end.append('24')
+            if start:
+                start = '\033[{}m'.format(';'.join(start))
+                end = '\033[{}m'.format(';'.join(end))
+            self.styles[token] = start or '', end or ''
+
+    def format(self, tokensource, outfile):
+        for ttype, value in tokensource:
+            not_found = True
+            if value != '\n':
+                while ttype and not_found:
+                    tok = self.styles.get(ttype)
+                    if tok is None:
+                        ttype = ttype[:-1]
+                    else:
+                        on, off = tok
+                        lines = value.split('\n')
+                        for line in lines:
+                            if line:
+                                outfile.write(on + line + off)
+                            if line is not lines[-1]:
+                                outfile.write('\n')
+                        not_found = False
+
+            if not_found:
+                outfile.write(value)
+
+
+formatter = None
+
+
+def initialize_highlighter(style='default'):
+    global formatter
+    formatter = DiffFormatter(style)
+
+
+def highlight_data(code, filename, aliases=None):
+    if aliases:
+        base, ext = os.path.splitext(filename)
+        alias = aliases.get(ext[1:])
+        if alias is not None:
+            filename = base + '.' + alias
+    try:
+        lexer = get_lexer_for_filename(filename, stripnl=False)
+    except ClassNotFound:
+        pass
+    else:
+        return highlight(code, lexer, formatter)
+
+
+split_pat = re.compile(r'(\033\[.*?m)')
+
+
+def highlight_line(line):
+    ans = []
+    current = None
+    pos = 0
+    for x in split_pat.split(line):
+        if x.startswith('\033'):
+            if current is None:
+                current = Segment(pos, x)
+            else:
+                current.end = pos
+                current.end_code = x
+                ans.append(current)
+                current = None
+        else:
+            pos += len(x)
+    return ans
+
+
+def highlight_for_diff(path, aliases):
+    ans = []
+    lines = lines_for_path(path)
+    hd = highlight_data('\n'.join(lines), path, aliases)
+    if hd is not None:
+        for line in hd.splitlines():
+            ans.append(highlight_line(line))
+    return ans
+
+
+def highlight_collection(collection, aliases=None):
+    jobs = {}
+    ans = {}
+    with concurrent.futures.ProcessPoolExecutor(max_workers=os.cpu_count()) as executor:
+        for path, item_type, other_path in collection:
+            if item_type != 'rename':
+                for p in (path, other_path):
+                    if p:
+                        is_binary = isinstance(data_for_path(p), bytes)
+                        if not is_binary:
+                            jobs[executor.submit(highlight_for_diff, p, aliases)] = p
+        for future in concurrent.futures.as_completed(jobs):
+            path = jobs[future]
+            try:
+                highlights = future.result()
+            except Exception as e:
+                return 'Running syntax highlighting for {} generated an exception: {}'.format(path, e)
+            ans[path] = highlights
+    return ans
+
+
+def main():
+    from .config import defaults
+    # kitty +runpy "from kittens.diff.highlight import main; main()" file
+    import sys
+    initialize_highlighter()
+    with open(sys.argv[-1]) as f:
+        highlighted = highlight_data(f.read(), f.name, defaults.syntax_aliases)
+    if highlighted is None:
+        raise SystemExit('Unknown filetype: {}'.format(sys.argv[-1]))
+    print(highlighted)

kittens/diff/main.py

@@ -0,0 +1,422 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import os
+import sys
+import warnings
+from collections import defaultdict
+from functools import partial
+from gettext import gettext as _
+
+from kitty.cli import CONFIG_HELP, appname, parse_args
+from kitty.fast_data_types import wcswidth
+from kitty.key_encoding import (
+    DOWN, END, ESCAPE, HOME, PAGE_DOWN, PAGE_UP, RELEASE, UP
+)
+
+from ..tui.handler import Handler
+from ..tui.images import ImageManager
+from ..tui.loop import Loop
+from ..tui.operations import styled
+from .collect import (
+    create_collection, data_for_path, lines_for_path, set_highlight_data
+)
+from .config import init_config
+from .patch import Differ, set_diff_command
+from .render import ImageSupportWarning, LineRef, render_diff
+
+try:
+    from .highlight import initialize_highlighter, highlight_collection
+except ImportError:
+    initialize_highlighter = None
+
+
+INITIALIZING, COLLECTED, DIFFED = range(3)
+
+
+def generate_diff(collection, context):
+    d = Differ()
+
+    for path, item_type, changed_path in collection:
+        if item_type == 'diff':
+            is_binary = isinstance(data_for_path(path), bytes)
+            if not is_binary:
+                d.add_diff(path, changed_path)
+
+    return d(context)
+
+
+class DiffHandler(Handler):
+
+    image_manager_class = ImageManager
+
+    def __init__(self, args, opts, left, right):
+        self.state = INITIALIZING
+        self.opts = opts
+        self.left, self.right = left, right
+        self.report_traceback_on_exit = None
+        self.args = args
+        self.scroll_pos = self.max_scroll_pos = 0
+        self.current_context_count = self.original_context_count = self.args.context
+        if self.current_context_count < 0:
+            self.current_context_count = self.original_context_count = self.opts.num_context_lines
+        self.highlighting_done = False
+        self.restore_position = None
+
+    def create_collection(self):
+        self.start_job('collect', create_collection, self.left, self.right)
+
+    def generate_diff(self):
+        self.start_job('diff', generate_diff, self.collection, self.current_context_count)
+
+    def calculate_statistics(self):
+        self.added_count = self.collection.added_count
+        self.removed_count = self.collection.removed_count
+        for patch in self.diff_map.values():
+            self.added_count += patch.added_count
+            self.removed_count += patch.removed_count
+
+    def render_diff(self):
+        self.diff_lines = tuple(render_diff(self.collection, self.diff_map, self.args, self.screen_size.cols, self.image_manager))
+        self.ref_path_map = defaultdict(list)
+        for i, l in enumerate(self.diff_lines):
+            self.ref_path_map[l.ref.path].append((i, l.ref))
+        self.max_scroll_pos = len(self.diff_lines) - self.num_lines
+
+    @property
+    def current_position(self):
+        return self.diff_lines[min(len(self.diff_lines) - 1, self.scroll_pos)].ref
+
+    @current_position.setter
+    def current_position(self, ref):
+        num = None
+        if isinstance(ref.extra, LineRef):
+            sln = ref.extra.src_line_number
+            for i, q in self.ref_path_map[ref.path]:
+                if isinstance(q.extra, LineRef):
+                    if q.extra.src_line_number >= sln:
+                        if q.extra.src_line_number == sln:
+                            num = i
+                        break
+                    num = i
+        if num is None:
+            for i, q in self.ref_path_map[ref.path]:
+                num = i
+                break
+
+        if num is not None:
+            self.scroll_pos = min(num, self.max_scroll_pos)
+
+    @property
+    def num_lines(self):
+        return self.screen_size.rows - 1
+
+    def scroll_to_next_change(self, backwards=False):
+        if backwards:
+            r = range(self.scroll_pos - 1, -1, -1)
+        else:
+            r = range(self.scroll_pos + 1, len(self.diff_lines))
+        for i in r:
+            line = self.diff_lines[i]
+            if line.is_change_start:
+                self.scroll_lines(i - self.scroll_pos)
+                return
+        self.cmd.bell()
+
+    def set_scrolling_region(self):
+        self.cmd.set_scrolling_region(self.screen_size, 0, self.num_lines - 2)
+
+    def scroll_lines(self, amt=1):
+        new_pos = max(0, min(self.scroll_pos + amt, self.max_scroll_pos))
+        if new_pos == self.scroll_pos:
+            self.cmd.bell()
+            return
+        if abs(new_pos - self.scroll_pos) >= self.num_lines - 1:
+            self.scroll_pos = new_pos
+            self.draw_screen()
+            return
+        self.enforce_cursor_state()
+        self.cmd.scroll_screen(amt)
+        self.scroll_pos = new_pos
+        if amt < 0:
+            self.cmd.set_cursor_position(0, 0)
+            self.draw_lines(-amt)
+        else:
+            self.cmd.set_cursor_position(0, self.num_lines - amt)
+            self.draw_lines(amt, self.num_lines - amt)
+        self.draw_status_line()
+
+    def init_terminal_state(self):
+        self.cmd.set_line_wrapping(False)
+        self.cmd.set_window_title(main.title)
+        self.cmd.set_default_colors(self.opts.foreground, self.opts.background)
+
+    def initialize(self):
+        self.init_terminal_state()
+        self.set_scrolling_region()
+        self.draw_screen()
+        self.create_collection()
+
+    def enforce_cursor_state(self):
+        self.cmd.set_cursor_visible(self.state > DIFFED)
+
+    def finalize(self):
+        self.cmd.set_cursor_visible(True)
+        self.cmd.set_default_colors()
+        self.cmd.set_scrolling_region()
+
+    def draw_lines(self, num, offset=0):
+        offset += self.scroll_pos
+        image_involved = False
+        limit = len(self.diff_lines)
+        for i in range(num):
+            lpos = offset + i
+            if lpos >= limit:
+                text = ''
+            else:
+                line = self.diff_lines[lpos]
+                text = line.text
+                if line.image_data is not None:
+                    image_involved = True
+            self.write('\r\x1b[K' + text + '\x1b[0m')
+            if i < num - 1:
+                self.write('\n')
+        if image_involved:
+            self.place_images()
+
+    def place_images(self):
+        self.cmd.clear_images_on_screen()
+        offset = self.scroll_pos
+        limit = len(self.diff_lines)
+        in_image = False
+        for row in range(self.num_lines):
+            lpos = offset + row
+            if lpos >= limit:
+                break
+            line = self.diff_lines[lpos]
+            if in_image:
+                if line.image_data is None:
+                    in_image = False
+                continue
+            if line.image_data is not None:
+                left_placement, right_placement = line.image_data
+                if left_placement is not None:
+                    self.place_image(row, left_placement, True)
+                    in_image = True
+                if right_placement is not None:
+                    self.place_image(row, right_placement, False)
+                    in_image = True
+
+    def place_image(self, row, placement, is_left):
+        xpos = (0 if is_left else (self.screen_size.cols // 2)) + placement.image.margin_size
+        image_height_in_rows = placement.image.rows
+        topmost_visible_row = placement.row
+        num_visible_rows = image_height_in_rows - topmost_visible_row
+        visible_frac = min(num_visible_rows / image_height_in_rows, 1)
+        if visible_frac > 0:
+            height = int(visible_frac * placement.image.height)
+            top = placement.image.height - height
+            self.image_manager.show_image(placement.image.image_id, xpos, row, src_rect=(
+                0, top, placement.image.width, height))
+
+    def draw_screen(self):
+        self.enforce_cursor_state()
+        if self.state < DIFFED:
+            self.cmd.clear_screen()
+            self.write(_('Calculating diff, please wait...'))
+            return
+        self.cmd.clear_images_on_screen()
+        self.cmd.set_cursor_position(0, 0)
+        self.draw_lines(self.num_lines)
+        self.draw_status_line()
+
+    def draw_status_line(self):
+        if self.state < DIFFED:
+            return
+        self.cmd.set_cursor_position(0, self.num_lines)
+        self.cmd.clear_to_eol()
+        scroll_frac = styled('{:.0%}'.format(self.scroll_pos / self.max_scroll_pos), fg=self.opts.margin_fg)
+        counts = '{}{}{}'.format(
+                styled(str(self.added_count), fg=self.opts.highlight_added_bg),
+                styled(',', fg=self.opts.margin_fg),
+                styled(str(self.removed_count), fg=self.opts.highlight_removed_bg)
+        )
+        suffix = counts + '  ' + scroll_frac
+        prefix = styled(':', fg=self.opts.margin_fg)
+        filler = self.screen_size.cols - wcswidth(prefix) - wcswidth(suffix)
+        text = '{}{}{}'.format(prefix, ' ' * filler, suffix)
+        self.write(text)
+
+    def change_context_count(self, new_ctx):
+        new_ctx = max(0, new_ctx)
+        if new_ctx != self.current_context_count:
+            self.current_context_count = new_ctx
+            self.state = COLLECTED
+            self.generate_diff()
+            self.restore_position = self.current_position
+            self.draw_screen()
+
+    def on_text(self, text, in_bracketed_paste=False):
+        if text == 'q':
+            if self.state <= DIFFED:
+                self.quit_loop(0)
+                return
+        if self.state is DIFFED:
+            if text in 'jk':
+                self.scroll_lines(1 if text == 'j' else -1)
+                return
+            if text in 'a+-=':
+                new_ctx = self.current_context_count
+                if text == 'a':
+                    new_ctx = 100000
+                elif text == '=':
+                    new_ctx = self.original_context_count
+                else:
+                    new_ctx += (-1 if text == '-' else 1) * 5
+                self.change_context_count(new_ctx)
+            if text in 'np':
+                self.scroll_to_next_change(backwards=text == 'p')
+                return
+
+    def on_key(self, key_event):
+        if key_event.type is RELEASE:
+            return
+        if key_event.key is ESCAPE:
+            if self.state <= DIFFED:
+                self.quit_loop(0)
+                return
+        if self.state is DIFFED:
+            if key_event.key is UP or key_event.key is DOWN:
+                self.scroll_lines(1 if key_event.key is DOWN else -1)
+                return
+            if key_event.key is PAGE_UP or key_event.key is PAGE_DOWN:
+                amt = self.num_lines * (1 if key_event.key is PAGE_DOWN else -1)
+                self.scroll_lines(amt)
+                return
+            if key_event.key is HOME or key_event.key is END:
+                amt = len(self.diff_lines) * (1 if key_event.key is END else -1)
+                self.scroll_lines(amt)
+                return
+
+    def on_resize(self, screen_size):
+        self.screen_size = screen_size
+        self.set_scrolling_region()
+        if self.state > COLLECTED:
+            self.image_manager.delete_all_sent_images()
+            self.render_diff()
+        self.draw_screen()
+
+    def on_job_done(self, job_id, job_result):
+        if 'tb' in job_result:
+            self.report_traceback_on_exit = job_result['tb']
+            self.quit_loop(1)
+            return
+        if job_id == 'collect':
+            self.collection = job_result['result']
+            self.state = COLLECTED
+            self.generate_diff()
+        elif job_id == 'diff':
+            diff_map = job_result['result']
+            if isinstance(diff_map, str):
+                self.report_traceback_on_exit = diff_map
+                self.quit_loop(1)
+                return
+            self.state = DIFFED
+            self.diff_map = diff_map
+            self.calculate_statistics()
+            self.render_diff()
+            self.scroll_pos = 0
+            if self.restore_position is not None:
+                self.current_position = self.restore_position
+                self.restore_position = None
+            self.draw_screen()
+            if initialize_highlighter is not None and not self.highlighting_done:
+                from .highlight import StyleNotFound
+                self.highlighting_done = True
+                try:
+                    initialize_highlighter(self.opts.pygments_style)
+                except StyleNotFound as e:
+                    self.report_traceback_on_exit = str(e)
+                    self.quit_loop(1)
+                    return
+                self.start_job('highlight', highlight_collection, self.collection, self.opts.syntax_aliases)
+        elif job_id == 'highlight':
+            hdata = job_result['result']
+            if isinstance(hdata, str):
+                self.report_traceback_on_exit = diff_map
+                self.quit_loop(1)
+                return
+            set_highlight_data(hdata)
+            self.render_diff()
+            self.draw_screen()
+
+    def on_interrupt(self):
+        self.quit_loop(1)
+
+    def on_eot(self):
+        self.quit_loop(1)
+
+
+OPTIONS = partial('''\
+--context
+type=int
+default=-1
+Number of lines of context to show between changes. Negative values
+use the number set in diff.conf
+
+
+--config
+type=list
+{config_help}
+
+
+--override -o
+type=list
+Override individual configuration options, can be specified multiple times.
+Syntax: |_ name=value|. For example: |_ -o background=gray|
+
+'''.format, config_help=CONFIG_HELP.format(conf_name='diff', appname=appname))
+
+
+def showwarning(message, category, filename, lineno, file=None, line=None):
+    if category is ImageSupportWarning:
+        showwarning.warnings.append(message)
+
+
+showwarning.warnings = []
+
+
+def main(args):
+    warnings.showwarning = showwarning
+    msg = 'Show a side-by-side diff of the specified files/directories'
+    args, items = parse_args(args[1:], OPTIONS, 'file_or_directory file_or_directory', msg, 'kitty +kitten diff')
+    if len(items) != 2:
+        raise SystemExit('You must specify exactly two files/directories to compare')
+    left, right = items
+    main.title = _('{} vs. {}').format(left, right)
+    if os.path.isdir(left) != os.path.isdir(right):
+        raise SystemExit('The items to be diffed should both be either directories or files. Comparing a directory to a file is not valid.')
+    opts = init_config(args)
+    set_diff_command(opts.diff_cmd)
+    lines_for_path.replace_tab_by = opts.replace_tab_by
+
+    loop = Loop()
+    handler = DiffHandler(args, opts, left, right)
+    loop.loop(handler)
+    for message in showwarning.warnings:
+        from kitty.utils import safe_print
+        safe_print(message, file=sys.stderr)
+    if loop.return_code != 0:
+        if handler.report_traceback_on_exit:
+            print(handler.report_traceback_on_exit, file=sys.stderr)
+            input('Press Enter to quit.')
+        raise SystemExit(loop.return_code)
+
+
+def handle_result(args, current_char, target_window_id, boss):
+    pass
+
+
+if __name__ == '__main__':
+    main(sys.argv)

kittens/diff/patch.py

@@ -0,0 +1,237 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import concurrent.futures
+import os
+import shlex
+import shutil
+import subprocess
+
+from .collect import lines_for_path
+from .diff_speedup import changed_center
+
+left_lines = right_lines = None
+GIT_DIFF = 'git diff --no-color --no-ext-diff --exit-code -U_CONTEXT_ --no-index --'
+DIFF_DIFF = 'diff -p -U _CONTEXT_ --'
+
+
+def find_differ():
+    if shutil.which('git'):
+        return GIT_DIFF
+    if shutil.which('diff'):
+        return DIFF_DIFF
+
+
+def set_diff_command(opt):
+    if opt == 'auto':
+        cmd = find_differ()
+        if cmd is None:
+            raise SystemExit('Failed to find either the git or diff programs on your system')
+    else:
+        cmd = opt
+    set_diff_command.cmd = cmd
+
+
+def run_diff(file1, file2, context=3):
+    # returns: ok, is_different, patch
+    cmd = shlex.split(set_diff_command.cmd.replace('_CONTEXT_', str(context)))
+    p = subprocess.Popen(
+            cmd + [file1, file2],
+            stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.DEVNULL)
+    stdout, stderr = p.communicate()
+    returncode = p.wait()
+    if returncode in (0, 1):
+        return True, returncode == 1, stdout.decode('utf-8')
+    return False, returncode, stderr.decode('utf-8')
+
+
+class Chunk:
+
+    __slots__ = ('is_context', 'left_start', 'right_start', 'left_count', 'right_count', 'centers')
+
+    def __init__(self, left_start, right_start, is_context=False):
+        self.is_context = is_context
+        self.left_start = left_start
+        self.right_start = right_start
+        self.left_count = self.right_count = 0
+        self.centers = None
+
+    def add_line(self):
+        self.right_count += 1
+
+    def remove_line(self):
+        self.left_count += 1
+
+    def context_line(self):
+        self.left_count += 1
+        self.right_count += 1
+
+    def finalize(self):
+        if not self.is_context and self.left_count == self.right_count:
+            self.centers = tuple(
+                changed_center(left_lines[self.left_start + i], right_lines[self.right_start + i])
+                for i in range(self.left_count)
+            )
+
+    def __repr__(self):
+        return 'Chunk(is_context={}, left_start={}, left_count={}, right_start={}, right_count={})'.format(
+                self.is_context, self.left_start, self.left_count, self.right_start, self.right_count)
+
+
+class Hunk:
+
+    def __init__(self, title, left, right):
+        self.left_start, self.left_count = left
+        self.right_start, self.right_count = right
+        self.left_start -= 1  # 0-index
+        self.right_start -= 1  # 0-index
+        self.title = title
+        self.added_count = self.removed_count = 0
+        self.chunks = []
+        self.current_chunk = None
+        self.largest_line_number = max(self.left_start + self.left_count, self.right_start + self.right_count)
+
+    def new_chunk(self, is_context=False):
+        if self.chunks:
+            c = self.chunks[-1]
+            left_start = c.left_start + c.left_count
+            right_start = c.right_start + c.right_count
+        else:
+            left_start = self.left_start
+            right_start = self.right_start
+        return Chunk(left_start, right_start, is_context)
+
+    def ensure_diff_chunk(self):
+        if self.current_chunk is None:
+            self.current_chunk = self.new_chunk(is_context=False)
+        elif self.current_chunk.is_context:
+            self.chunks.append(self.current_chunk)
+            self.current_chunk = self.new_chunk(is_context=False)
+
+    def ensure_context_chunk(self):
+        if self.current_chunk is None:
+            self.current_chunk = self.new_chunk(is_context=True)
+        elif not self.current_chunk.is_context:
+            self.chunks.append(self.current_chunk)
+            self.current_chunk = self.new_chunk(is_context=True)
+
+    def add_line(self):
+        self.ensure_diff_chunk()
+        self.current_chunk.add_line()
+        self.added_count += 1
+
+    def remove_line(self):
+        self.ensure_diff_chunk()
+        self.current_chunk.remove_line()
+        self.removed_count += 1
+
+    def context_line(self):
+        self.ensure_context_chunk()
+        self.current_chunk.context_line()
+
+    def finalize(self):
+        self.chunks.append(self.current_chunk)
+        del self.current_chunk
+        # Sanity check
+        c = self.chunks[-1]
+        if c.left_start + c.left_count != self.left_start + self.left_count:
+            raise ValueError('Left side line mismatch {} != {}'.format(c.left_start + c.left_count, self.left_start + self.left_count))
+        if c.right_start + c.right_count != self.right_start + self.right_count:
+            raise ValueError('Left side line mismatch {} != {}'.format(c.right_start + c.right_count, self.right_start + self.right_count))
+        for c in self.chunks:
+            c.finalize()
+
+
+def parse_range(x):
+    parts = x[1:].split(',', 1)
+    start = abs(int(parts[0]))
+    count = 1 if len(parts) < 2 else int(parts[1])
+    return start, count
+
+
+def parse_hunk_header(line):
+    parts = tuple(filter(None, line.split('@@', 2)))
+    linespec = parts[0].strip()
+    title = ''
+    if len(parts) == 2:
+        title = parts[1].strip()
+    left, right = map(parse_range, linespec.split())
+    return Hunk(title, left, right)
+
+
+class Patch:
+
+    def __init__(self, all_hunks):
+        self.all_hunks = all_hunks
+        self.largest_line_number = self.all_hunks[-1].largest_line_number if self.all_hunks else 0
+        self.added_count = sum(h.added_count for h in all_hunks)
+        self.removed_count = sum(h.removed_count for h in all_hunks)
+
+    def __iter__(self):
+        return iter(self.all_hunks)
+
+    def __len__(self):
+        return len(self.all_hunks)
+
+
+def parse_patch(raw):
+    all_hunks = []
+    current_hunk = None
+    for line in raw.splitlines():
+        if line.startswith('@@ '):
+            current_hunk = parse_hunk_header(line)
+            all_hunks.append(current_hunk)
+        else:
+            if current_hunk is None:
+                continue
+            q = line[0]
+            if q == '+':
+                current_hunk.add_line()
+            elif q == '-':
+                current_hunk.remove_line()
+            elif q == '\\':
+                continue
+            else:
+                current_hunk.context_line()
+    for h in all_hunks:
+        h.finalize()
+    return Patch(all_hunks)
+
+
+class Differ:
+
+    def __init__(self):
+        self.jmap = {}
+        self.jobs = []
+
+    def add_diff(self, file1, file2):
+        self.jmap[file1] = file2
+        self.jobs.append(file1)
+
+    def __call__(self, context=3):
+        global left_lines, right_lines
+        ans = {}
+        with concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count()) as executor:
+            jobs = {executor.submit(run_diff, key, self.jmap[key], context): key for key in self.jobs}
+            for future in concurrent.futures.as_completed(jobs):
+                key = jobs[future]
+                left_path, right_path = key, self.jmap[key]
+                try:
+                    ok, returncode, output = future.result()
+                except FileNotFoundError as err:
+                    return 'Could not find the {} executable. Is it in your PATH?'.format(err.filename)
+                except Exception as e:
+                    return 'Running git diff for {} vs. {} generated an exception: {}'.format(left_path, right_path, e)
+                if not ok:
+                    return output + '\nRunning git diff for {} vs. {} failed'.format(left_path, right_path)
+                left_lines = lines_for_path(left_path)
+                right_lines = lines_for_path(right_path)
+                try:
+                    patch = parse_patch(output)
+                except Exception:
+                    import traceback
+                    return traceback.format_exc() + '\nParsing diff for {} vs. {} failed'.format(left_path, right_path)
+                else:
+                    ans[key] = patch
+        return ans

kittens/diff/render.py

@@ -0,0 +1,489 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import warnings
+from gettext import gettext as _
+from itertools import repeat, zip_longest
+from math import ceil
+
+from kitty.fast_data_types import truncate_point_for_length, wcswidth
+
+from ..tui.images import can_display_images, screen_size
+from .collect import (
+    Segment, data_for_path, highlights_for_path, is_image, lines_for_path,
+    path_name_map, sanitize
+)
+from .config import formats
+from .diff_speedup import split_with_highlights as _split_with_highlights
+
+
+class ImageSupportWarning(Warning):
+    pass
+
+
+def images_supported():
+    ans = getattr(images_supported, 'ans', None)
+    if ans is None:
+        images_supported.ans = ans = can_display_images()
+        if not ans:
+            warnings.warn('ImageMagick not found images cannot be displayed', ImageSupportWarning)
+    return ans
+
+
+class Ref:
+
+    def __setattr__(self, name, value):
+        raise AttributeError("can't set attribute")
+
+    def __repr__(self):
+        return '{}({})'.format(self.__class__.__name__, ', '.join(
+            '{}={}'.format(n, getattr(self, n)) for n in self.__slots__ if n != '_hash'))
+
+
+class LineRef(Ref):
+
+    __slots__ = ('src_line_number', 'wrapped_line_idx')
+
+    def __init__(self, sln, wli=0):
+        object.__setattr__(self, 'src_line_number', sln)
+        object.__setattr__(self, 'wrapped_line_idx', wli)
+
+
+class Reference(Ref):
+
+    __slots__ = ('path', 'extra')
+
+    def __init__(self, path, extra=None):
+        object.__setattr__(self, 'path', path)
+        object.__setattr__(self, 'extra', extra)
+
+
+class Line:
+
+    __slots__ = ('text', 'ref', 'is_change_start', 'image_data')
+
+    def __init__(self, text, ref, change_start=False, image_data=None):
+        self.text = text
+        self.ref = ref
+        self.is_change_start = change_start
+        self.image_data = image_data
+
+
+def yield_lines_from(iterator, reference, is_change_start=True):
+    for text in iterator:
+        yield Line(text, reference, is_change_start)
+        is_change_start = False
+
+
+def human_readable(size, sep=' '):
+    """ Convert a size in bytes into a human readable form """
+    divisor, suffix = 1, "B"
+    for i, candidate in enumerate(('B', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB')):
+        if size < (1 << ((i + 1) * 10)):
+            divisor, suffix = (1 << (i * 10)), candidate
+            break
+    size = str(float(size)/divisor)
+    if size.find(".") > -1:
+        size = size[:size.find(".")+2]
+    if size.endswith('.0'):
+        size = size[:-2]
+    return size + sep + suffix
+
+
+def fit_in(text, count):
+    p = truncate_point_for_length(text, count)
+    if p >= len(text):
+        return text
+    if count > 1:
+        p = truncate_point_for_length(text, count - 1)
+    return text[:p] + '…'
+
+
+def fill_in(text, sz):
+    w = wcswidth(text)
+    if w < sz:
+        text += ' ' * (sz - w)
+    return text
+
+
+def place_in(text, sz):
+    return fill_in(fit_in(text, sz), sz)
+
+
+def format_func(which):
+    def formatted(text):
+        fmt = formats[which]
+        return '\x1b[' + fmt + 'm' + text + '\x1b[0m'
+    formatted.__name__ = which + '_format'
+    return formatted
+
+
+text_format = format_func('text')
+title_format = format_func('title')
+margin_format = format_func('margin')
+added_format = format_func('added')
+removed_format = format_func('removed')
+removed_margin_format = format_func('removed_margin')
+added_margin_format = format_func('added_margin')
+filler_format = format_func('filler')
+hunk_margin_format = format_func('hunk_margin')
+hunk_format = format_func('hunk')
+highlight_map = {'remove': ('removed_highlight', 'removed'), 'add': ('added_highlight', 'added')}
+
+
+def highlight_boundaries(ltype):
+    s, e = highlight_map[ltype]
+    start = '\x1b[' + formats[s] + 'm'
+    stop = '\x1b[' + formats[e] + 'm'
+    return start, stop
+
+
+def title_lines(left_path, right_path, args, columns, margin_size):
+    m = ' ' * margin_size
+    left_name, right_name = map(path_name_map.get, (left_path, right_path))
+    if right_name and right_name != left_name:
+        n1 = fit_in(m + sanitize(left_name), columns // 2 - margin_size)
+        n1 = place_in(n1, columns // 2)
+        n2 = fit_in(m + sanitize(right_name), columns // 2 - margin_size)
+        n2 = place_in(n2, columns // 2)
+        name = n1 + n2
+    else:
+        name = place_in(m + sanitize(left_name), columns)
+    yield title_format(place_in(name, columns))
+    yield title_format('━' * columns)
+
+
+def binary_lines(path, other_path, columns, margin_size):
+    template = _('Binary file: {}')
+    available_cols = columns // 2 - margin_size
+
+    def fl(path, fmt):
+        text = template.format(human_readable(len(data_for_path(path))))
+        text = place_in(text, available_cols)
+        return margin_format(' ' * margin_size) + fmt(text)
+
+    if path is None:
+        filler = render_diff_line('', '', 'filler', margin_size, available_cols)
+        yield filler + fl(other_path, added_format)
+    elif other_path is None:
+        filler = render_diff_line('', '', 'filler', margin_size, available_cols)
+        yield fl(path, removed_format) + filler
+    else:
+        yield fl(path, removed_format) + fl(other_path, added_format)
+
+
+def split_to_size(line, width):
+    if not line:
+        yield line
+    while line:
+        p = truncate_point_for_length(line, width)
+        yield line[:p]
+        line = line[p:]
+
+
+def truncate_points(line, width):
+    pos = 0
+    sz = len(line)
+    while True:
+        pos = truncate_point_for_length(line, width, pos)
+        if pos < sz:
+            yield pos
+        else:
+            break
+
+
+def split_with_highlights(line, width, highlights, bg_highlight=None):
+    truncate_pts = list(truncate_points(line, width))
+    return _split_with_highlights(line, truncate_pts, highlights, bg_highlight)
+
+
+margin_bg_map = {'filler': filler_format, 'remove': removed_margin_format, 'add': added_margin_format, 'context': margin_format}
+text_bg_map = {'filler': filler_format, 'remove': removed_format, 'add': added_format, 'context': text_format}
+
+
+class DiffData:
+
+    def __init__(self, left_path, right_path, available_cols, margin_size):
+        self.left_path, self.right_path = left_path, right_path
+        self.available_cols = available_cols
+        self.margin_size = margin_size
+        self.left_lines, self.right_lines = map(lines_for_path, (left_path, right_path))
+        self.filler_line = render_diff_line('', '', 'filler', margin_size, available_cols)
+        self.left_filler_line = render_diff_line('', '', 'remove', margin_size, available_cols)
+        self.right_filler_line = render_diff_line('', '', 'add', margin_size, available_cols)
+        self.left_hdata = highlights_for_path(left_path)
+        self.right_hdata = highlights_for_path(right_path)
+
+    def left_highlights_for_line(self, line_num):
+        if line_num < len(self.left_hdata):
+            return self.left_hdata[line_num]
+        return []
+
+    def right_highlights_for_line(self, line_num):
+        if line_num < len(self.right_hdata):
+            return self.right_hdata[line_num]
+        return []
+
+
+def render_diff_line(number, text, ltype, margin_size, available_cols):
+    margin = margin_bg_map[ltype](place_in(number, margin_size))
+    content = text_bg_map[ltype](fill_in(text or '', available_cols))
+    return margin + content
+
+
+def render_diff_pair(left_line_number, left, left_is_change, right_line_number, right, right_is_change, is_first, margin_size, available_cols):
+    ltype = 'filler' if left_line_number is None else ('remove' if left_is_change else 'context')
+    rtype = 'filler' if right_line_number is None else ('add' if right_is_change else 'context')
+    return (
+            render_diff_line(left_line_number if is_first else None, left, ltype, margin_size, available_cols) +
+            render_diff_line(right_line_number if is_first else None, right, rtype, margin_size, available_cols)
+    )
+
+
+def hunk_title(hunk_num, hunk, margin_size, available_cols):
+    m = hunk_margin_format(' ' * margin_size)
+    t = '@@ -{},{} +{},{} @@ {}'.format(hunk.left_start + 1, hunk.left_count, hunk.right_start + 1, hunk.right_count, hunk.title)
+    return m + hunk_format(place_in(t, available_cols))
+
+
+def render_half_line(line_number, line, highlights, ltype, margin_size, available_cols, changed_center=None):
+    bg_highlight = None
+    if changed_center is not None and changed_center[0]:
+        prefix_count, suffix_count = changed_center
+        line_sz = len(line)
+        if prefix_count + suffix_count < line_sz:
+            start, stop = highlight_boundaries(ltype)
+            seg = Segment(prefix_count, start)
+            seg.end = line_sz - suffix_count
+            seg.end_code = stop
+            bg_highlight = seg
+    if highlights or bg_highlight:
+        lines = split_with_highlights(line, available_cols, highlights, bg_highlight)
+    else:
+        lines = split_to_size(line, available_cols)
+    line_number = str(line_number + 1)
+    for line in lines:
+        yield render_diff_line(line_number, line, ltype, margin_size, available_cols)
+        line_number = ''
+
+
+def lines_for_chunk(data, hunk_num, chunk, chunk_num):
+    if chunk.is_context:
+        for i in range(chunk.left_count):
+            left_line_number = line_ref = chunk.left_start + i
+            right_line_number = chunk.right_start + i
+            highlights = data.left_highlights_for_line(left_line_number)
+            if highlights:
+                lines = split_with_highlights(data.left_lines[left_line_number], data.available_cols, highlights)
+            else:
+                lines = split_to_size(data.left_lines[left_line_number], data.available_cols)
+            left_line_number = str(left_line_number + 1)
+            right_line_number = str(right_line_number + 1)
+            for wli, text in enumerate(lines):
+                line = render_diff_line(left_line_number, text, 'context', data.margin_size, data.available_cols)
+                if right_line_number == left_line_number:
+                    r = line
+                else:
+                    r = render_diff_line(right_line_number, text, 'context', data.margin_size, data.available_cols)
+                ref = Reference(data.left_path, LineRef(line_ref, wli))
+                yield Line(line + r, ref)
+                left_line_number = right_line_number = ''
+    else:
+        common = min(chunk.left_count, chunk.right_count)
+        for i in range(max(chunk.left_count, chunk.right_count)):
+            ll, rl = [], []
+            if i < chunk.left_count:
+                rln = ref_ln = chunk.left_start + i
+                ll.extend(render_half_line(
+                    rln, data.left_lines[rln], data.left_highlights_for_line(rln),
+                    'remove', data.margin_size, data.available_cols,
+                    None if chunk.centers is None else chunk.centers[i]))
+                ref_path = data.left_path
+            if i < chunk.right_count:
+                rln = ref_ln = chunk.right_start + i
+                rl.extend(render_half_line(
+                    rln, data.right_lines[rln], data.right_highlights_for_line(rln),
+                    'add', data.margin_size, data.available_cols,
+                    None if chunk.centers is None else chunk.centers[i]))
+                ref_path = data.right_path
+            if i < common:
+                extra = len(ll) - len(rl)
+                if extra != 0:
+                    if extra < 0:
+                        x, fl = ll, data.left_filler_line
+                        extra = -extra
+                    else:
+                        x, fl = rl, data.right_filler_line
+                    x.extend(repeat(fl, extra))
+            else:
+                if ll:
+                    x, count = rl, len(ll)
+                else:
+                    x, count = ll, len(rl)
+                x.extend(repeat(data.filler_line, count))
+            for wli, (left_line, right_line) in enumerate(zip(ll, rl)):
+                ref = Reference(ref_path, LineRef(ref_ln, wli))
+                yield Line(left_line + right_line, ref, i == 0 and wli == 0)
+
+
+def lines_for_diff(left_path, right_path, hunks, args, columns, margin_size):
+    available_cols = columns // 2 - margin_size
+    data = DiffData(left_path, right_path, available_cols, margin_size)
+
+    for hunk_num, hunk in enumerate(hunks):
+        yield Line(hunk_title(hunk_num, hunk, margin_size, columns - margin_size), Reference(left_path, LineRef(hunk.left_start)))
+        for cnum, chunk in enumerate(hunk.chunks):
+            yield from lines_for_chunk(data, hunk_num, chunk, cnum)
+
+
+def all_lines(path, args, columns, margin_size, is_add=True):
+    available_cols = columns // 2 - margin_size
+    ltype = 'add' if is_add else 'remove'
+    lines = lines_for_path(path)
+    filler = render_diff_line('', '', 'filler', margin_size, available_cols)
+    msg_written = False
+    hdata = highlights_for_path(path)
+
+    def highlights(num):
+        return hdata[num] if num < len(hdata) else []
+
+    for line_number, line in enumerate(lines):
+        h = render_half_line(line_number, line, highlights(line_number), ltype, margin_size, available_cols)
+        for i, hl in enumerate(h):
+            ref = Reference(path, LineRef(line_number, i))
+            empty = filler
+            if not msg_written:
+                msg_written = True
+                empty = render_diff_line(
+                        '', _('This file was added') if is_add else _('This file was removed'),
+                        'filler', margin_size, available_cols)
+            text = (empty + hl) if is_add else (hl + empty)
+            yield Line(text, ref, line_number == 0 and i == 0)
+
+
+def rename_lines(path, other_path, args, columns, margin_size):
+    m = ' ' * margin_size
+    for line in split_to_size(_('The file {0} was renamed to {1}').format(
+            sanitize(path_name_map[path]), sanitize(path_name_map[other_path])), columns - margin_size):
+        yield m + line
+
+
+class Image:
+
+    def __init__(self, image_id, width, height, margin_size):
+        self.image_id = image_id
+        self.width, self.height = width, height
+        ss = screen_size()
+        self.rows = int(ceil(self.height / ss.cell_height))
+        self.columns = int(ceil(self.width / ss.cell_width))
+        self.margin_size = margin_size
+
+
+class ImagePlacement:
+
+    def __init__(self, image, row):
+        self.image = image
+        self.row = row
+
+
+def render_image(path, is_left, available_cols, margin_size, image_manager):
+    lnum = 0
+    margin_fmt = removed_margin_format if is_left else added_margin_format
+    m = margin_fmt(' ' * margin_size)
+    fmt = removed_format if is_left else added_format
+
+    def yield_split(text):
+        nonlocal lnum
+        for i, line in enumerate(split_to_size(text, available_cols)):
+            yield m + fmt(place_in(line, available_cols)), Reference(path, LineRef(lnum, i)), None
+        lnum += 1
+
+    try:
+        image_id, width, height = image_manager.send_image(path, available_cols - margin_size, screen_size().rows - 2)
+    except Exception as e:
+        yield from yield_split(_('Failed to render image, with error:'))
+        yield from yield_split(' '.join(str(e).splitlines()))
+        return
+    meta = _('Dimensions: {0}x{1} pixels Size: {2}').format(
+            width, height, human_readable(len(data_for_path(path))))
+    yield from yield_split(meta)
+    bg_line = m + fmt(' ' * available_cols)
+    img = Image(image_id, width, height, margin_size)
+    for r in range(img.rows):
+        yield bg_line, Reference(path, LineRef(lnum)), ImagePlacement(img, r)
+        lnum += 1
+
+
+def image_lines(left_path, right_path, columns, margin_size, image_manager):
+    available_cols = columns // 2 - margin_size
+    left_lines, right_lines = iter(()), iter(())
+    if left_path is not None:
+        left_lines = render_image(left_path, True, available_cols, margin_size, image_manager)
+    if right_path is not None:
+        right_lines = render_image(right_path, False, available_cols, margin_size, image_manager)
+    filler = ' ' * (available_cols + margin_size)
+    is_change_start = True
+    for left, right in zip_longest(left_lines, right_lines):
+        left_placement = right_placement = None
+        if left is None:
+            left = filler
+            right, ref, right_placement = right
+        elif right is None:
+            right = filler
+            left, ref, left_placement = left
+        else:
+            right, ref, right_placement = right
+            left, ref, left_placement = left
+        image_data = (left_placement, right_placement) if left_placement or right_placement else None
+        yield Line(left + right, ref, is_change_start, image_data)
+        is_change_start = False
+
+
+def render_diff(collection, diff_map, args, columns, image_manager):
+    largest_line_number = 0
+    for path, item_type, other_path in collection:
+        if item_type == 'diff':
+            patch = diff_map.get(path)
+            if patch is not None:
+                largest_line_number = max(largest_line_number, patch.largest_line_number)
+
+    margin_size = max(3, len(str(largest_line_number)) + 1)
+    last_item_num = len(collection) - 1
+
+    for i, (path, item_type, other_path) in enumerate(collection):
+        item_ref = Reference(path)
+        is_binary = isinstance(data_for_path(path), bytes)
+        is_img = is_binary and is_image(path) and images_supported()
+        yield from yield_lines_from(title_lines(path, other_path, args, columns, margin_size), item_ref, False)
+        if item_type == 'diff':
+            if is_binary:
+                if is_img:
+                    ans = image_lines(path, other_path, columns, margin_size, image_manager)
+                else:
+                    ans = yield_lines_from(binary_lines(path, other_path, columns, margin_size), item_ref)
+            else:
+                ans = lines_for_diff(path, other_path, diff_map[path], args, columns, margin_size)
+        elif item_type == 'add':
+            if is_binary:
+                if is_img:
+                    ans = image_lines(None, path, columns, margin_size, image_manager)
+                else:
+                    ans = yield_lines_from(binary_lines(None, path, columns, margin_size), item_ref)
+            else:
+                ans = all_lines(path, args, columns, margin_size, is_add=True)
+        elif item_type == 'removal':
+            if is_binary:
+                if is_img:
+                    ans = image_lines(path, None, columns, margin_size, image_manager)
+                else:
+                    ans = yield_lines_from(binary_lines(path, None, columns, margin_size), item_ref)
+            else:
+                ans = all_lines(path, args, columns, margin_size, is_add=False)
+        elif item_type == 'rename':
+            ans = yield_lines_from(rename_lines(path, other_path, args, columns, margin_size), item_ref)
+        else:
+            raise ValueError('Unsupported item type: {}'.format(item_type))
+        yield from ans
+        if i < last_item_num:
+            yield Line('', item_ref)

kittens/diff/speedup.c

@@ -0,0 +1,179 @@
+/*
+ * speedup.c
+ * Copyright (C) 2018 Kovid Goyal <kovid at kovidgoyal.net>
+ *
+ * Distributed under terms of the GPL3 license.
+ */
+
+#include "data-types.h"
+
+static PyObject*
+changed_center(PyObject *self UNUSED, PyObject *args) {
+    unsigned int prefix_count = 0, suffix_count = 0;
+    PyObject *lp, *rp;
+    if (!PyArg_ParseTuple(args, "UU", &lp, &rp)) return NULL;
+    const size_t left_len = PyUnicode_GET_LENGTH(lp), right_len = PyUnicode_GET_LENGTH(rp);
+
+#define R(which, index) PyUnicode_READ(PyUnicode_KIND(which), PyUnicode_DATA(which), index)
+    while(prefix_count < MIN(left_len, right_len)) {
+        if (R(lp, prefix_count) != R(rp, prefix_count)) break;
+        prefix_count++;
+    }
+    if (left_len && right_len && prefix_count < MIN(left_len, right_len)) {
+        while(suffix_count < MIN(left_len - prefix_count, right_len - prefix_count)) {
+            if(R(lp, left_len - 1 - suffix_count) != R(rp, right_len - 1 - suffix_count)) break;
+            suffix_count++;
+        }
+    }
+#undef R
+    return Py_BuildValue("II", prefix_count, suffix_count);
+}
+
+typedef struct {
+    unsigned int start_pos, end_pos, current_pos;
+    PyObject *start_code, *end_code;
+} Segment;
+
+typedef struct {
+    Segment sg;
+    unsigned int num, pos;
+} SegmentPointer;
+
+static const Segment EMPTY_SEGMENT = { .current_pos = UINT_MAX };
+
+static inline bool
+convert_segment(PyObject *highlight, Segment *dest) {
+    PyObject *val = NULL;
+#define I
+#define A(x, d, c) { \
+    val = PyObject_GetAttrString(highlight, #x); \
+    if (val == NULL) return false; \
+    dest->d = c(val); Py_DECREF(val); \
+}
+    A(start, start_pos, PyLong_AsUnsignedLong);
+    A(end, end_pos, PyLong_AsUnsignedLong);
+    dest->current_pos = dest->start_pos;
+    A(start_code, start_code, I);
+    A(end_code, end_code, I);
+    if (!PyUnicode_Check(dest->start_code)) { PyErr_SetString(PyExc_TypeError, "start_code is not a string"); return false; }
+    if (!PyUnicode_Check(dest->end_code)) { PyErr_SetString(PyExc_TypeError, "end_code is not a string"); return false; }
+#undef A
+#undef I
+    return true;
+}
+
+static inline bool
+next_segment(SegmentPointer *s, PyObject *highlights) {
+    if (s->pos < s->num) {
+        if (!convert_segment(PyList_GET_ITEM(highlights, s->pos), &s->sg)) return false;
+        s->pos++;
+    } else s->sg.current_pos = UINT_MAX;
+    return true;
+}
+
+static inline bool
+insert_code(PyObject *code, Py_UCS4 *buf, size_t bufsz, unsigned int *buf_pos) {
+    unsigned int csz = PyUnicode_GET_LENGTH(code);
+    if (*buf_pos + csz >= bufsz) return false;
+    for (unsigned int s = 0; s < csz; s++) buf[(*buf_pos)++] = PyUnicode_READ(PyUnicode_KIND(code), PyUnicode_DATA(code), s);
+    return true;
+}
+
+static inline bool
+add_line(Segment *bg_segment, Segment *fg_segment, Py_UCS4 *buf, size_t bufsz, unsigned int *buf_pos, PyObject *ans) {
+    bool bg_is_active = bg_segment->current_pos == bg_segment->end_pos, fg_is_active = fg_segment->current_pos == fg_segment->end_pos;
+    if (bg_is_active) { if(!insert_code(bg_segment->end_code, buf, bufsz, buf_pos)) return false; }
+    if (fg_is_active) { if(!insert_code(fg_segment->end_code, buf, bufsz, buf_pos)) return false; }
+    PyObject *wl = PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, buf, *buf_pos);
+    if (!wl) return false;
+    int ret = PyList_Append(ans, wl); Py_DECREF(wl); if (ret != 0) return false;
+    *buf_pos = 0;
+    if (bg_is_active) { if(!insert_code(bg_segment->start_code, buf, bufsz, buf_pos)) return false; }
+    if (fg_is_active) { if(!insert_code(fg_segment->start_code, buf, bufsz, buf_pos)) return false; }
+    return true;
+}
+
+static PyObject*
+split_with_highlights(PyObject *self UNUSED, PyObject *args) {
+    PyObject *line, *truncate_points_py, *fg_highlights, *bg_highlight;
+    if (!PyArg_ParseTuple(args, "UO!O!O", &line, &PyList_Type, &truncate_points_py, &PyList_Type, &fg_highlights, &bg_highlight)) return NULL;
+    PyObject *ans = PyList_New(0);
+    if (!ans) return NULL;
+    static unsigned int truncate_points[256];
+    unsigned int num_truncate_pts = PyList_GET_SIZE(truncate_points_py), truncate_pos = 0, truncate_point;
+    for (unsigned int i = 0; i < MIN(num_truncate_pts, arraysz(truncate_points)); i++) {
+        truncate_points[i] = PyLong_AsUnsignedLong(PyList_GET_ITEM(truncate_points_py, i));
+    }
+    SegmentPointer fg_segment = { .sg = EMPTY_SEGMENT, .num = PyList_GET_SIZE(fg_highlights)}, bg_segment = { .sg = EMPTY_SEGMENT };
+    if (bg_highlight != Py_None) { if (!convert_segment(bg_highlight, &bg_segment.sg)) { Py_CLEAR(ans); return NULL; }; bg_segment.num = 1; }
+#define CHECK_CALL(func, ...) if (!func(__VA_ARGS__)) { Py_CLEAR(ans); if (!PyErr_Occurred()) PyErr_SetString(PyExc_ValueError, "line too long"); return NULL; }
+    CHECK_CALL(next_segment, &fg_segment, fg_highlights);
+
+#define NEXT_TRUNCATE_POINT truncate_point = (truncate_pos < num_truncate_pts) ? truncate_points[truncate_pos++] : UINT_MAX
+    NEXT_TRUNCATE_POINT;
+
+#define INSERT_CODE(x) { CHECK_CALL(insert_code, x, buf, arraysz(buf), &buf_pos); }
+
+#define ADD_LINE CHECK_CALL(add_line, &bg_segment.sg, &fg_segment.sg, buf, arraysz(buf), &buf_pos, ans);
+
+#define ADD_CHAR(x) { \
+    buf[buf_pos++] = x; \
+    if (buf_pos >= arraysz(buf)) { Py_CLEAR(ans); PyErr_SetString(PyExc_ValueError, "line too long"); return NULL; } \
+}
+#define CHECK_SEGMENT(sgp, is_fg) { \
+    if (i == sgp.sg.current_pos) { \
+        INSERT_CODE(sgp.sg.current_pos == sgp.sg.start_pos ? sgp.sg.start_code : sgp.sg.end_code); \
+        if (sgp.sg.current_pos == sgp.sg.start_pos) sgp.sg.current_pos = sgp.sg.end_pos; \
+        else { \
+            if (is_fg) { \
+                CHECK_CALL(next_segment, &fg_segment, fg_highlights); \
+                if (sgp.sg.current_pos == i) { \
+                    INSERT_CODE(sgp.sg.start_code); \
+                    sgp.sg.current_pos = sgp.sg.end_pos; \
+                } \
+            } else sgp.sg.current_pos = UINT_MAX; \
+        } \
+    }\
+}
+
+    const unsigned int line_sz = PyUnicode_GET_LENGTH(line);
+    static Py_UCS4 buf[4096];
+    unsigned int i = 0, buf_pos = 0;
+    for (; i < line_sz; i++) {
+        if (i == truncate_point) { ADD_LINE; NEXT_TRUNCATE_POINT; }
+        CHECK_SEGMENT(bg_segment, false);
+        CHECK_SEGMENT(fg_segment, true)
+        ADD_CHAR(PyUnicode_READ(PyUnicode_KIND(line), PyUnicode_DATA(line), i));
+    }
+    if (buf_pos) ADD_LINE;
+    return ans;
+#undef INSERT_CODE
+#undef CHECK_SEGMENT
+#undef CHECK_CALL
+#undef ADD_CHAR
+#undef ADD_LINE
+#undef NEXT_TRUNCATE_POINT
+}
+
+static PyMethodDef module_methods[] = {
+    {"changed_center", (PyCFunction)changed_center, METH_VARARGS, ""},
+    {"split_with_highlights", (PyCFunction)split_with_highlights, METH_VARARGS, ""},
+    {NULL, NULL, 0, NULL}        /* Sentinel */
+};
+
+static struct PyModuleDef module = {
+   .m_base = PyModuleDef_HEAD_INIT,
+   .m_name = "diff_speedup",   /* name of module */
+   .m_doc = NULL,
+   .m_size = -1,
+   .m_methods = module_methods
+};
+
+EXPORTED PyMODINIT_FUNC
+PyInit_diff_speedup(void) {
+    PyObject *m;
+
+    m = PyModule_Create(&module);
+    if (m == NULL) return NULL;
+    return m;
+}

kittens/hints/main.py

@@ -0,0 +1,389 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import os
+import re
+import string
+import sys
+from functools import lru_cache, partial
+from gettext import gettext as _
+
+from kitty.cli import parse_args
+from kitty.fast_data_types import set_clipboard_string
+from kitty.key_encoding import ESCAPE, backspace_key, enter_key
+from kitty.utils import screen_size_function
+
+from ..tui.handler import Handler
+from ..tui.loop import Loop
+from ..tui.operations import faint, styled
+
+URL_PREFIXES = 'http https file ftp'.split()
+HINT_ALPHABET = string.digits + string.ascii_lowercase
+screen_size = screen_size_function()
+
+
+class Mark(object):
+
+    __slots__ = ('index', 'start', 'end', 'text')
+
+    def __init__(self, index, start, end, text):
+        self.index, self.start, self.end = index, start, end
+        self.text = text
+
+
+@lru_cache(maxsize=2048)
+def encode_hint(num):
+    res = ''
+    d = len(HINT_ALPHABET)
+    while not res or num > 0:
+        num, i = divmod(num, d)
+        res = HINT_ALPHABET[i] + res
+    return res
+
+
+def decode_hint(x):
+    return int(x, 36)
+
+
+def highlight_mark(m, text, current_input):
+    hint = encode_hint(m.index)
+    if current_input and not hint.startswith(current_input):
+        return faint(text)
+    hint = hint[len(current_input):] or ' '
+    text = text[len(hint):]
+    return styled(
+        hint,
+        fg='black',
+        bg='green',
+        bold=True
+    ) + styled(
+        text, fg='gray', fg_intense=True, bold=True
+    )
+
+
+def render(text, current_input, all_marks):
+    for mark in reversed(all_marks):
+        mtext = highlight_mark(mark, text[mark.start:mark.end], current_input)
+        text = text[:mark.start] + mtext + text[mark.end:]
+
+    text = text.replace('\0', '')
+
+    return text.replace('\n', '\r\n').rstrip()
+
+
+class Hints(Handler):
+
+    def __init__(self, text, all_marks, index_map, args):
+        self.text, self.index_map = text, index_map
+        self.all_marks = all_marks
+        self.current_input = ''
+        self.current_text = None
+        self.args = args
+        self.window_title = _('Choose URL') if args.type == 'url' else _('Choose text')
+        self.chosen = None
+
+    def init_terminal_state(self):
+        self.cmd.set_cursor_visible(False)
+        self.cmd.set_window_title(self.window_title)
+        self.cmd.set_line_wrapping(False)
+
+    def initialize(self):
+        self.init_terminal_state()
+        self.draw_screen()
+
+    def on_text(self, text, in_bracketed_paste):
+        changed = False
+        for c in text:
+            if c in HINT_ALPHABET:
+                self.current_input += c
+                changed = True
+        if changed:
+            matches = [
+                m.text for idx, m in self.index_map.items()
+                if encode_hint(idx).startswith(self.current_input)
+            ]
+            if len(matches) == 1:
+                self.chosen = matches[0]
+                self.quit_loop(0)
+                return
+            self.current_text = None
+            self.draw_screen()
+
+    def on_key(self, key_event):
+        if key_event is backspace_key:
+            self.current_input = self.current_input[:-1]
+            self.current_text = None
+            self.draw_screen()
+        elif key_event is enter_key and self.current_input:
+            try:
+                idx = decode_hint(self.current_input)
+                self.chosen = self.index_map[idx].text
+            except Exception:
+                self.current_input = ''
+                self.current_text = None
+                self.draw_screen()
+            else:
+                self.quit_loop(0)
+        elif key_event.key is ESCAPE:
+            self.quit_loop(1)
+
+    def on_interrupt(self):
+        self.quit_loop(1)
+
+    def on_eot(self):
+        self.quit_loop(1)
+
+    def on_resize(self, new_size):
+        Handler.on_resize(self, new_size)
+        self.draw_screen()
+
+    def draw_screen(self):
+        if self.current_text is None:
+            self.current_text = render(self.text, self.current_input, self.all_marks)
+        self.cmd.clear_screen()
+        self.write(self.current_text)
+
+
+def regex_finditer(pat, minimum_match_length, text):
+    for m in pat.finditer(text):
+        s, e = m.span(pat.groups)
+        while e > s + 1 and text[e-1] == '\0':
+            e -= 1
+        if e - s >= minimum_match_length:
+            yield s, e
+
+
+closing_bracket_map = {'(': ')', '[': ']', '{': '}', '<': '>'}
+opening_brackets = ''.join(closing_bracket_map)
+postprocessor_map = {}
+
+
+def postprocessor(func):
+    postprocessor_map[func.__name__] = func
+    return func
+
+
+@postprocessor
+def url(text, s, e):
+    if s > 4 and text[s - 5:s] == 'link:':  # asciidoc URLs
+        url = text[s:e]
+        idx = url.rfind('[')
+        if idx > -1:
+            e -= len(url) - idx
+    while text[e - 1] in '.,?!' and e > 1:  # remove trailing punctuation
+        e -= 1
+    # Remove trailing bracket if matched by leading bracket
+    if s > 0 and e < len(text) and text[s-1] in opening_brackets and text[e-1] == closing_bracket_map[text[s-1]]:
+        e -= 1
+    # Remove trailing quote if matched by leading quote
+    if s > 0 and e < len(text) and text[s-1] in '\'"' and text[e-1] == text[s-1]:
+        e -= 1
+    return s, e
+
+
+@postprocessor
+def brackets(text, s, e):
+    # Remove matching brackets
+    if e > s and e <= len(text):
+        before = text[s]
+        if before in '({[<' and text[e-1] == closing_bracket_map[before]:
+            s += 1
+            e -= 1
+    return s, e
+
+
+@postprocessor
+def quotes(text, s, e):
+    # Remove matching quotes
+    if e > s and e <= len(text):
+        before = text[s]
+        if before in '\'"' and text[e-1] == before:
+            s += 1
+            e -= 1
+    return s, e
+
+
+def mark(pattern, post_processors, text, args):
+    pat = re.compile(pattern)
+    for idx, (s, e) in enumerate(regex_finditer(pat, args.minimum_match_length, text)):
+        for func in post_processors:
+            s, e = func(text, s, e)
+        mark_text = text[s:e].replace('\n', '').replace('\0', '')
+        yield Mark(idx, s, e, mark_text)
+
+
+def run_loop(args, text, all_marks, index_map):
+    loop = Loop()
+    handler = Hints(text, all_marks, index_map, args)
+    loop.loop(handler)
+    if handler.chosen and loop.return_code == 0:
+        return {'match': handler.chosen, 'program': args.program}
+    raise SystemExit(loop.return_code)
+
+
+def escape(chars):
+    return chars.replace('\\', '\\\\').replace('-', r'\-').replace(']', r'\]')
+
+
+def functions_for(args):
+    post_processors = []
+    if args.type == 'url':
+        from .url_regex import url_delimiters
+        pattern = '(?:{})://[^{}]{{3,}}'.format(
+            '|'.join(args.url_prefixes.split(',')), url_delimiters
+        )
+        post_processors.append(url)
+    elif args.type == 'path':
+        pattern = r'(?:\S*/\S+)|(?:\S+[.][a-zA-Z0-9]{2,5})'
+        post_processors.extend((brackets, quotes))
+    elif args.type == 'line':
+        pattern = '(?m)^\\s*(.+)[\\s\0]*$'
+    elif args.type == 'word':
+        chars = args.word_characters
+        if chars is None:
+            import json
+            chars = json.loads(os.environ['KITTY_COMMON_OPTS'])['select_by_word_characters']
+        pattern = r'(?u)[{}\w]{{{},}}'.format(escape(chars), args.minimum_match_length)
+        post_processors.extend((brackets, quotes))
+    else:
+        pattern = args.regex
+    return pattern, post_processors
+
+
+def convert_text(text, cols):
+    lines = []
+    for full_line in text.split('\n'):
+        if full_line:
+            for line in full_line.split('\r'):
+                if line:
+                    lines.append(line.ljust(cols, '\0'))
+    return '\n'.join(lines)
+
+
+def parse_input(text):
+    try:
+        cols = int(os.environ['OVERLAID_WINDOW_COLS'])
+    except KeyError:
+        cols = screen_size().cols
+    return convert_text(text, cols)
+
+
+def run(args, text):
+    try:
+        pattern, post_processors = functions_for(args)
+        text = parse_input(text)
+        all_marks = tuple(mark(pattern, post_processors, text, args))
+        if not all_marks:
+            input(_('No {} found, press Enter to quit.').format(
+                'URLs' if args.type == 'url' else 'matches'
+                ))
+            return
+
+        largest_index = all_marks[-1].index
+        for m in all_marks:
+            m.index = largest_index - m.index
+        index_map = {m.index: m for m in all_marks}
+    except Exception:
+        import traceback
+        traceback.print_exc()
+        input('Press Enter to quit.')
+        raise SystemExit(1)
+
+    return run_loop(args, text, all_marks, index_map)
+
+
+# CLI {{{
+OPTIONS = partial(r'''
+--program
+default=default
+What program to use to open matched text. Defaults to the default open program
+for the operating system.  Use a value of - to paste the match into the
+terminal window instead. A value of @ will copy the match to the clipboard.
+
+
+--type
+default=url
+choices=url,regex,path,line,word
+The type of text to search for.
+
+
+--regex
+default=(?m)^\s*(.+)\s*$
+The regular expression to use when --type=regex.  If you specify a group in the
+regular expression only the group will be matched. This allow you to match text
+ignoring a prefix/suffix, as needed. The default expression matches lines.
+
+
+--url-prefixes
+default={0}
+Comma separated list of recognized URL prefixes.
+
+
+--word-characters
+Characters to consider as part of a word. In addition, all characters marked as
+alpha-numeric in the unicode database will be considered as word characters.
+Defaults to the select_by_word_characters setting from kitty.conf.
+
+
+--minimum-match-length
+default=3
+type=int
+The minimum number of characters to consider a match.
+'''.format, ','.join(sorted(URL_PREFIXES)))
+
+
+def parse_hints_args(args):
+    msg = 'Select text from the screen using the keyboard. Defaults to searching for URLs.'
+    return parse_args(args, OPTIONS, '', msg, 'hints')
+
+
+def main(args):
+    text = ''
+    if sys.stdin.isatty():
+        if '--help' not in args and '-h' not in args:
+            print('You must pass the text to be hinted on STDIN', file=sys.stderr)
+            input(_('Press Enter to quit'))
+            return
+    else:
+        text = sys.stdin.buffer.read().decode('utf-8')
+        sys.stdin = open('/dev/tty')
+    try:
+        args, items = parse_hints_args(args[1:])
+    except SystemExit as e:
+        if e.code != 0:
+            print(e.args[0], file=sys.stderr)
+            input(_('Press Enter to quit'))
+        return
+    if items:
+        print('Extra command line arguments present: {}'.format(' '.join(items)), file=sys.stderr)
+        input(_('Press Enter to quit'))
+        return
+    return run(args, text)
+
+
+def handle_result(args, data, target_window_id, boss):
+    program = data['program']
+    if program == '-':
+        w = boss.window_id_map.get(target_window_id)
+        if w is not None:
+            w.paste(data['match'])
+    elif program == '@':
+        set_clipboard_string(data['match'])
+    else:
+        cwd = None
+        w = boss.window_id_map.get(target_window_id)
+        if w is not None:
+            cwd = w.cwd_of_child
+        boss.open_url(data['match'], None if program == 'default' else program, cwd=cwd)
+
+
+handle_result.type_of_input = 'screen'
+
+
+if __name__ == '__main__':
+    # Run with kitty +kitten hints
+    ans = main(sys.argv)
+    if ans:
+        print(ans)
+# }}}

kittens/hints/url_regex.py

@@ -0,0 +1 @@
+url_delimiters = '\x00-\x09\x0b-\x20\x7f-\xa0\xad\u0600-\u0605\u061c\u06dd\u070f\u08e2\u1680\u180e\u2000-\u200f\u2028-\u202f\u205f-\u2064\u2066-\u206f\u3000\ud800-\uf8ff\ufeff\ufff9-\ufffb\U000110bd\U0001bca0-\U0001bca3\U0001d173-\U0001d17a\U000e0001\U000e0020-\U000e007f\U000f0000-\U000ffffd\U00100000-\U0010fffd'  # noqa

kittens/icat/main.py

@@ -2,41 +2,26 @@
 # vim:fileencoding=utf-8
 # License: GPL v3 Copyright: 2017, Kovid Goyal <kovid at kovidgoyal.net>
 
-import array
-import codecs
-import fcntl
 import mimetypes
 import os
 import re
 import signal
-import subprocess
 import sys
-import termios
 import zlib
 from base64 import standard_b64encode
 from collections import namedtuple
-from math import ceil, floor
+from math import ceil
 from tempfile import NamedTemporaryFile
 
 from kitty.cli import parse_args
 from kitty.constants import appname
-from kitty.utils import read_with_timeout
-
-try:
-    fsenc = sys.getfilesystemencoding() or 'utf-8'
-    codecs.lookup(fsenc)
-except Exception:
-    fsenc = 'utf-8'
-
-
-class OpenFailed(ValueError):
-
-    def __init__(self, path, message):
-        ValueError.__init__(
-            self, 'Failed to open: {} with error: {}'.format(path, message)
-        )
-        self.path = path
+from kitty.utils import fit_image, read_with_timeout
 
+from ..tui.images import (
+    ConvertFailed, NoImageMagick, OpenFailed, convert, fsenc, identify,
+    screen_size
+)
+from ..tui.operations import clear_images_on_screen, serialize_gr_command
 
 OPTIONS = '''\
 --align
@@ -100,46 +85,11 @@ def options_spec():
     return options_spec.ans
 
 
-Size = namedtuple('Size', 'rows cols width height')
-
-
-def screen_size():
-    if screen_size.changed:
-        buf = array.array('H', [0, 0, 0, 0])
-        fcntl.ioctl(sys.stdout, termios.TIOCGWINSZ, buf)
-        screen_size.ans = Size(*buf)
-        screen_size.changed = False
-    return screen_size.ans
-
-
-screen_size.changed = True
-
-
 def write_gr_cmd(cmd, payload=None):
-    cmd = ','.join('{}={}'.format(k, v) for k, v in cmd.items())
-    w = sys.stdout.buffer.write
-    w(b'\033_G'), w(cmd.encode('ascii'))
-    if payload:
-        w(b';')
-        w(payload)
-    w(b'\033\\')
+    sys.stdout.buffer.write(serialize_gr_command(cmd, payload))
     sys.stdout.flush()
 
 
-def fit_image(width, height, pwidth, pheight):
-    if height > pheight:
-        corrf = pheight / float(height)
-        width, height = floor(corrf * width), pheight
-    if width > pwidth:
-        corrf = pwidth / float(width)
-        width, height = pwidth, floor(corrf * height)
-    if height > pheight:
-        corrf = pheight / float(height)
-        width, height = floor(corrf * width), pheight
-
-    return int(width), int(height)
-
-
 def calculate_in_cell_x_offset(width, cell_width, align):
     if align == 'left':
         return 0
@@ -217,52 +167,6 @@ def show(outfile, width, height, fmt, transmit_mode='t', align='center', place=N
         write_chunked(cmd, data)
 
 
-ImageData = namedtuple('ImageData', 'fmt width height mode')
-
-
-def run_imagemagick(path, cmd, keep_stdout=True):
-    try:
-        p = subprocess.run(cmd, stdout=subprocess.PIPE if keep_stdout else subprocess.DEVNULL, stderr=subprocess.PIPE)
-    except FileNotFoundError:
-        raise SystemExit('ImageMagick is required to cat images')
-    if p.returncode != 0:
-        raise OpenFailed(path, p.stderr.decode('utf-8'))
-    return p
-
-
-def identify(path):
-    p = run_imagemagick(path, ['identify', '-format', '%m %w %h %A', path])
-    parts = tuple(filter(None, p.stdout.decode('utf-8').split()))
-    mode = 'rgb' if parts[3].lower() == 'false' else 'rgba'
-    return ImageData(parts[0].lower(), int(parts[1]), int(parts[2]), mode)
-
-
-def convert(path, m, available_width, available_height, scale_up):
-    width, height = m.width, m.height
-    cmd = ['convert', '-background', 'none', path]
-    if scale_up:
-        if width < available_width:
-            r = available_width / width
-            width, height = available_width, int(height * r)
-    if width > available_width or height > available_height:
-        width, height = fit_image(width, height, available_width, available_height)
-        cmd += ['-resize', '{}x{}'.format(width, height)]
-    with NamedTemporaryFile(prefix='icat-', suffix='.' + m.mode, delete=False) as outfile:
-        run_imagemagick(path, cmd + [outfile.name])
-    # ImageMagick sometimes generated rgba images smaller than the specified
-    # size. See https://github.com/kovidgoyal/kitty/issues/276 for examples
-    sz = os.path.getsize(outfile.name)
-    bytes_per_pixel = 3 if m.mode == 'rgb' else 4
-    expected_size = bytes_per_pixel * width * height
-    if sz < expected_size:
-        missing = expected_size - sz
-        if missing % (bytes_per_pixel * width) != 0:
-            raise SystemExit('ImageMagick failed to convert {} correctly, it generated {} < {} of data'.format(path, sz, expected_size))
-        height -= missing // (bytes_per_pixel * width)
-
-    return outfile.name, width, height
-
-
 def process(path, args):
     m = identify(path)
     ss = screen_size()
@@ -369,7 +273,7 @@ def main(args=sys.argv):
         detect_support.has_files = args.transfer_mode == 'file'
     errors = []
     if args.clear:
-        write_gr_cmd({'a': 'd'})
+        sys.stdout.buffer.write(clear_images_on_screen(delete_data=True))
         if not items:
             return
     if not items:
@@ -385,6 +289,10 @@ def main(args=sys.argv):
                     process(item, args)
             else:
                 process(item, args)
+        except NoImageMagick as e:
+            raise SystemExit(str(e))
+        except ConvertFailed as e:
+            raise SystemExit(str(e))
         except OpenFailed as e:
             errors.append(e)
     if args.place:
@@ -396,5 +304,9 @@ def main(args=sys.argv):
     raise SystemExit(1)
 
 
+def handle_result(args, current_char, target_window_id, boss):
+    pass
+
+
 if __name__ == '__main__':
     main()

kittens/resize_window/main.py

@@ -0,0 +1,135 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+
+import sys
+
+from kitty.cli import parse_args
+from kitty.cmds import cmap, parse_subcommand_cli
+from kitty.constants import version
+from kitty.key_encoding import CTRL, ESCAPE, RELEASE, N, S, T, W
+from kitty.remote_control import encode_send, parse_rc_args
+
+from ..tui.handler import Handler
+from ..tui.loop import Loop
+from ..tui.operations import styled
+
+global_opts = None
+
+
+class Resize(Handler):
+
+    print_on_fail = None
+
+    def __init__(self, opts):
+        self.opts = opts
+
+    def initialize(self):
+        global global_opts
+        global_opts = parse_rc_args(['kitty', '@resize-window'])[0]
+        self.original_size = self.screen_size
+        self.cmd.set_cursor_visible(False)
+        self.cmd.set_line_wrapping(False)
+        self.draw_screen()
+
+    def do_window_resize(self, is_decrease=False, is_horizontal=True, reset=False, multiplier=1):
+        resize_window = cmap['resize-window']
+        increment = self.opts.horizontal_increment if is_horizontal else self.opts.vertical_increment
+        increment *= multiplier
+        if is_decrease:
+            increment *= -1
+        axis = 'reset' if reset else ('horizontal' if is_horizontal else 'vertical')
+        cmdline = [resize_window.name, '--self', '--increment={}'.format(increment), '--axis=' + axis]
+        opts, items = parse_subcommand_cli(resize_window, cmdline)
+        payload = resize_window(global_opts, opts, items)
+        send = {'cmd': resize_window.name, 'version': version, 'payload': payload}
+        self.write(encode_send(send))
+
+    def on_kitty_cmd_response(self, response):
+        if not response.get('ok'):
+            err = response['error']
+            if response.get('tb'):
+                err += '\n' + response['tb']
+            self.print_on_fail = err
+            self.quit_loop(1)
+            return
+        res = response.get('data')
+        if res:
+            self.cmd.bell()
+
+    def on_text(self, text, in_bracketed_paste=False):
+        text = text.upper()
+        if text in 'WNTSR':
+            self.do_window_resize(is_decrease=text in 'NS', is_horizontal=text in 'WN', reset=text == 'R')
+        elif text == 'Q':
+            self.quit_loop(0)
+
+    def on_key(self, key_event):
+        if key_event.type is RELEASE:
+            return
+        if key_event.key is ESCAPE:
+            self.quit_loop(0)
+        elif key_event.key in (W, N, T, S) and key_event.mods & CTRL:
+            self.do_window_resize(is_decrease=key_event.key in (N, S), is_horizontal=key_event.key in (W, N), multiplier=2)
+
+    def on_resize(self, new_size):
+        Handler.on_resize(self, new_size)
+        self.draw_screen()
+
+    def draw_screen(self):
+        self.cmd.clear_screen()
+        print = self.print
+        print(styled('Resize this window', bold=True, fg='gray', fg_intense=True))
+        print()
+        print('Press one of the following keys:')
+        print('  {}ider'.format(styled('W', fg='green')))
+        print('  {}arrower'.format(styled('N', fg='green')))
+        print('  {}aller'.format(styled('T', fg='green')))
+        print('  {}horter'.format(styled('S', fg='green')))
+        print('  {}eset'.format(styled('R', fg='red')))
+        print()
+        print('Press {} to quit resize mode'.format(styled('Esc', italic=True)))
+        print('Hold down {} to double step size'.format(styled('Ctrl', italic=True)))
+        print()
+        print(styled('Sizes', bold=True, fg='white', fg_intense=True))
+        print('Original: {} rows {} cols'.format(self.original_size.rows, self.original_size.cols))
+        print('Current:  {} rows {} cols'.format(
+            styled(self.screen_size.rows, fg='magenta'), styled(self.screen_size.cols, fg='magenta')))
+
+
+OPTIONS = r'''
+--horizontal-increment
+default=2
+type=int
+The base horizontal increment.
+
+
+--vertical-increment
+default=2
+type=int
+The base vertical increment.
+'''.format
+
+
+def main(args):
+    msg = 'Resize the current window'
+    try:
+        args, items = parse_args(args[1:], OPTIONS, '', msg, 'resize_window')
+    except SystemExit as e:
+        if e.code != 0:
+            print(e.args[0], file=sys.stderr)
+            input('Press Enter to quit')
+        return
+
+    loop = Loop()
+    handler = Resize(args)
+    loop.loop(handler)
+    if handler.print_on_fail:
+        print(handler.print_on_fail)
+        input('Press Enter to quit')
+    raise SystemExit(loop.return_code)
+
+
+def handle_result(args, data, target_window_id, boss):
+    pass

kittens/runner.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+
+import importlib
+import os
+import sys
+from functools import partial
+
+aliases = {'url_hints': 'hints'}
+
+
+def resolved_kitten(k):
+    return aliases.get(k, k)
+
+
+def import_kitten_main_module(config_dir, kitten):
+    if kitten.endswith('.py'):
+        path = os.path.expanduser(kitten)
+        if not os.path.isabs(path):
+            path = os.path.join(config_dir, path)
+        path = os.path.abspath(path)
+        if os.path.dirname(path):
+            sys.path.insert(0, os.path.dirname(path))
+        with open(path) as f:
+            src = f.read()
+        code = compile(src, path, 'exec')
+        g = {'__name__': 'kitten'}
+        exec(code, g)
+        return {'start': g['main'], 'end': g['handle_result']}
+    else:
+        kitten = resolved_kitten(kitten)
+        m = importlib.import_module('kittens.{}.main'.format(kitten))
+        return {'start': m.main, 'end': m.handle_result}
+
+
+def create_kitten_handler(kitten, orig_args):
+    from kitty.constants import config_dir
+    kitten = resolved_kitten(kitten)
+    m = import_kitten_main_module(config_dir, kitten)
+    ans = partial(m['end'], [kitten] + orig_args)
+    ans.type_of_input = getattr(m['end'], 'type_of_input', None)
+    return ans
+
+
+def set_debug(kitten):
+    from kittens.tui.loop import debug
+    import builtins
+    builtins.debug = debug
+
+
+def launch(args):
+    config_dir, kitten = args[:2]
+    kitten = resolved_kitten(kitten)
+    del args[:2]
+    args = [kitten] + args
+    os.environ['KITTY_CONFIG_DIRECTORY'] = config_dir
+    from kittens.tui.operations import clear_screen, reset_mode
+    set_debug(kitten)
+    m = import_kitten_main_module(config_dir, kitten)
+    try:
+        result = m['start'](args)
+    finally:
+        sys.stdin = sys.__stdin__
+    print(reset_mode('ALTERNATE_SCREEN') + clear_screen(), end='')
+    if result is not None:
+        import json
+        data = json.dumps(result)
+        print('OK:', len(data), data)
+    sys.stderr.flush()
+    sys.stdout.flush()
+
+
+def deserialize(output):
+    import json
+    if output.startswith('OK: '):
+        prefix, sz, rest = output.split(' ', 2)
+        return json.loads(rest[:int(sz)])
+
+
+def run_kitten(kitten):
+    import runpy
+    kitten = resolved_kitten(kitten)
+    set_debug(kitten)
+    runpy.run_module('kittens.{}.main'.format(kitten), run_name='__main__')
+
+
+def main():
+    try:
+        args = sys.argv[1:]
+        launch(args)
+    except Exception:
+        print('Unhandled exception running kitten:')
+        import traceback
+        traceback.print_exc()
+        input('Press Enter to quit...')

kittens/tui/handler.py

@@ -3,11 +3,39 @@
 # License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
 
 
+from .operations import commander
+
+
 class Handler:
 
-    def initialize(self, screen_size, quit_loop, wakeup):
+    image_manager_class = None
+
+    def _initialize(self, screen_size, quit_loop, wakeup, start_job, debug, image_manager=None):
         self.screen_size, self.quit_loop = screen_size, quit_loop
         self.wakeup = wakeup
+        self.debug = debug
+        self.start_job = start_job
+        self.cmd = commander(self)
+        self.image_manager = image_manager
+
+    def __enter__(self):
+        if self.image_manager is not None:
+            self.image_manager.__enter__()
+        self.debug.fobj = self
+        self.initialize()
+
+    def __exit__(self, *a):
+        del self.write_buf[:]
+        del self.debug.fobj
+        self.finalize()
+        if self.image_manager is not None:
+            self.image_manager.__exit__(*a)
+
+    def initialize(self):
+        pass
+
+    def finalize(self):
+        pass
 
     def on_resize(self, screen_size):
         self.screen_size = screen_size
@@ -30,6 +58,15 @@ class Handler:
     def on_eot(self):
         pass
 
+    def on_wakeup(self):
+        pass
+
+    def on_job_done(self, job_id, job_result):
+        pass
+
+    def on_kitty_cmd_response(self, response):
+        pass
+
     def write(self, data):
         if isinstance(data, str):
             data = data.encode('utf-8')

kittens/tui/images.py

@@ -0,0 +1,256 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import codecs
+import os
+import sys
+from base64 import standard_b64encode
+from collections import defaultdict, deque
+from itertools import count
+
+from kitty.utils import fit_image, screen_size_function
+
+from .operations import cursor
+
+try:
+    fsenc = sys.getfilesystemencoding() or 'utf-8'
+    codecs.lookup(fsenc)
+except Exception:
+    fsenc = 'utf-8'
+screen_size = screen_size_function()
+
+
+class ImageData:
+
+    def __init__(self, fmt, width, height, mode):
+        self.width, self.height, self.fmt, self.mode = width, height, fmt, mode
+        self.transmit_fmt = str(24 if self.mode == 'rgb' else 32)
+
+
+class OpenFailed(ValueError):
+
+    def __init__(self, path, message):
+        ValueError.__init__(
+            self, 'Failed to open image: {} with error: {}'.format(path, message)
+        )
+        self.path = path
+
+
+class ConvertFailed(ValueError):
+
+    def __init__(self, path, message):
+        ValueError.__init__(
+            self, 'Failed to convert image: {} with error: {}'.format(path, message)
+        )
+        self.path = path
+
+
+class NoImageMagick(Exception):
+    pass
+
+
+def run_imagemagick(path, cmd, keep_stdout=True):
+    import subprocess
+    try:
+        p = subprocess.run(cmd, stdout=subprocess.PIPE if keep_stdout else subprocess.DEVNULL, stderr=subprocess.PIPE)
+    except FileNotFoundError:
+        raise NoImageMagick('ImageMagick is required to process images')
+    if p.returncode != 0:
+        raise OpenFailed(path, p.stderr.decode('utf-8'))
+    return p
+
+
+def identify(path):
+    p = run_imagemagick(path, ['identify', '-format', '%m %w %h %A', path])
+    parts = tuple(filter(None, p.stdout.decode('utf-8').split()))
+    mode = 'rgb' if parts[3].lower() == 'false' else 'rgba'
+    return ImageData(parts[0].lower(), int(parts[1]), int(parts[2]), mode)
+
+
+def convert(path, m, available_width, available_height, scale_up, tdir=None):
+    from tempfile import NamedTemporaryFile
+    width, height = m.width, m.height
+    cmd = ['convert', '-background', 'none', path]
+    if scale_up:
+        if width < available_width:
+            r = available_width / width
+            width, height = available_width, int(height * r)
+    if width > available_width or height > available_height:
+        width, height = fit_image(width, height, available_width, available_height)
+        cmd += ['-resize', '{}x{}'.format(width, height)]
+    with NamedTemporaryFile(prefix='icat-', suffix='.' + m.mode, delete=False, dir=tdir) as outfile:
+        run_imagemagick(path, cmd + [outfile.name])
+    # ImageMagick sometimes generated rgba images smaller than the specified
+    # size. See https://github.com/kovidgoyal/kitty/issues/276 for examples
+    sz = os.path.getsize(outfile.name)
+    bytes_per_pixel = 3 if m.mode == 'rgb' else 4
+    expected_size = bytes_per_pixel * width * height
+    if sz < expected_size:
+        missing = expected_size - sz
+        if missing % (bytes_per_pixel * width) != 0:
+            raise ConvertFailed(path, 'ImageMagick failed to convert {} correctly, it generated {} < {} of data'.format(path, sz, expected_size))
+        height -= missing // (bytes_per_pixel * width)
+
+    return outfile.name, width, height
+
+
+def can_display_images():
+    import shutil
+    ans = getattr(can_display_images, 'ans', None)
+    if ans is None:
+        ans = shutil.which('convert') is not None
+        can_display_images.ans = ans
+    return ans
+
+
+class ImageManager:
+
+    def __init__(self, handler):
+        self.image_id_counter = count()
+        self.handler = handler
+        self.filesystem_ok = None
+        self.image_data = {}
+        self.failed_images = {}
+        self.converted_images = {}
+        self.sent_images = {}
+        self.image_id_to_image_data = {}
+        self.image_id_to_converted_data = {}
+        self.transmission_status = {}
+        self.placements_in_flight = defaultdict(deque)
+
+    @property
+    def next_image_id(self):
+        return next(self.image_id_counter) + 2
+
+    def __enter__(self):
+        import tempfile
+        self.tdir = tempfile.mkdtemp(prefix='kitten-images-')
+        with tempfile.NamedTemporaryFile(dir=self.tdir, delete=False) as f:
+            f.write(b'abcd')
+        self.handler.cmd.gr_command(dict(a='q', s=1, v=1, i=1, t='f'), standard_b64encode(f.name.encode(fsenc)))
+
+    def __exit__(self, *a):
+        import shutil
+        shutil.rmtree(self.tdir, ignore_errors=True)
+        self.handler.cmd.clear_images_on_screen(delete_data=True)
+        self.delete_all_sent_images()
+        del self.handler
+
+    def delete_all_sent_images(self):
+        for img_id in self.transmission_status:
+            self.handler.cmd.gr_command({'a': 'd', 'i': img_id})
+        self.transmission_status.clear()
+
+    def handle_response(self, apc):
+        cdata, payload = apc[1:].partition(';')[::2]
+        control = {}
+        for x in cdata.split(','):
+            k, v = x.partition('=')[::2]
+            control[k] = v
+        try:
+            image_id = int(control.get('i', '0'))
+        except Exception:
+            image_id = 0
+        if image_id == 1:
+            self.filesystem_ok = payload == 'OK'
+            return
+        if not image_id:
+            return
+        if not self.transmission_status.get(image_id):
+            self.transmission_status[image_id] = payload
+        else:
+            in_flight = self.placements_in_flight[image_id]
+            if in_flight:
+                pl = in_flight.popleft()
+                if payload.startswith('ENOENT:'):
+                    try:
+                        self.resend_image(image_id, pl)
+                    except Exception:
+                        pass
+                if not in_flight:
+                    self.placements_in_flight.pop(image_id, None)
+
+    def resend_image(self, image_id, pl):
+        image_data = self.image_id_to_image_data[image_id]
+        skey = self.image_id_to_converted_data[image_id]
+        self.transmit_image(image_data, image_id, *skey)
+        with cursor(self.handler.write):
+            self.handler.cmd.set_cursor_position(pl['x'], pl['y'])
+            self.handler.cmd.gr_command(pl['cmd'])
+
+    def send_image(self, path, max_cols=None, max_rows=None, scale_up=False):
+        path = os.path.abspath(path)
+        if path in self.failed_images:
+            raise self.failed_images[path]
+        if path not in self.image_data:
+            try:
+                self.image_data[path] = identify(path)
+            except Exception as e:
+                self.failed_images[path] = e
+                raise
+        m = self.image_data[path]
+        ss = screen_size()
+        if max_cols is None:
+            max_cols = ss.cols
+        if max_rows is None:
+            max_rows = ss.rows
+        available_width = max_cols * ss.cell_width
+        available_height = max_rows * ss.cell_height
+        key = path, available_width, available_height
+        skey = self.converted_images.get(key)
+        if skey is None:
+            try:
+                self.converted_images[key] = skey = self.convert_image(path, available_width, available_height, m, scale_up)
+            except Exception as e:
+                self.failed_images[path] = e
+                raise
+        final_width, final_height = skey[1:]
+        if final_width == 0:
+            return 0, 0, 0
+        image_id = self.sent_images.get(skey)
+        if image_id is None:
+            image_id = self.next_image_id
+            self.transmit_image(m, image_id, *skey)
+            self.sent_images[skey] = image_id
+        self.image_id_to_converted_data[image_id] = skey
+        self.image_id_to_image_data[image_id] = m
+        return image_id, skey[1], skey[2]
+
+    def hide_image(self, image_id):
+        self.handler.cmd.gr_command({'a': 'd', 'i': image_id})
+
+    def show_image(self, image_id, x, y, src_rect=None):
+        cmd = {'a': 'p', 'i': image_id}
+        if src_rect is not None:
+            cmd['x'], cmd['y'], cmd['w'], cmd['h'] = map(int, src_rect)
+        self.placements_in_flight[image_id].append({'cmd': cmd, 'x': x, 'y': y})
+        with cursor(self.handler.write):
+            self.handler.cmd.set_cursor_position(x, y)
+            self.handler.cmd.gr_command(cmd)
+
+    def convert_image(self, path, available_width, available_height, image_data, scale_up=False):
+        rgba_path, width, height = convert(path, image_data, available_width, available_height, scale_up, tdir=self.tdir)
+        return rgba_path, width, height
+
+    def transmit_image(self, image_data, image_id, rgba_path, width, height):
+        self.transmission_status[image_id] = 0
+        cmd = {'a': 't', 'f': image_data.transmit_fmt, 's': width, 'v': height, 'i': image_id}
+        if self.filesystem_ok:
+            cmd['t'] = 'f'
+            self.handler.cmd.gr_command(
+                cmd, standard_b64encode(rgba_path.encode(fsenc)))
+        else:
+            import zlib
+            data = open(rgba_path, 'rb').read()
+            cmd['S'] = len(data)
+            data = zlib.compress(data)
+            cmd['o'] = 'z'
+            data = standard_b64encode(data)
+            while data:
+                chunk, data = data[:4096], data[4096:]
+                m = 1 if data else 0
+                cmd['m'] = m
+                self.handler.cmd.gr_command(cmd, chunk)
+                cmd.clear()
+        return image_id

kittens/tui/loop.py

@@ -15,25 +15,32 @@ import tty
 from collections import namedtuple
 from contextlib import closing, contextmanager
 from functools import partial
+from queue import Empty, Queue
 
 from kitty.fast_data_types import parse_input_from_terminal, safe_pipe
-from kitty.icat import screen_size
 from kitty.key_encoding import (
     ALT, CTRL, PRESS, RELEASE, REPEAT, SHIFT, C, D, backspace_key,
     decode_key_event, enter_key
 )
+from kitty.utils import screen_size_function
 
 from .handler import Handler
-from .operations import init_state, reset_state, clear_screen
+from .operations import init_state, reset_state
+
+screen_size = screen_size_function()
 
 
-def log(*a, **kw):
-    fd = getattr(log, 'fd', None)
-    if fd is None:
-        fd = log.fd = open('/tmp/kitten-debug', 'w')
-    kw['file'] = fd
+def debug(*a, **kw):
+    from base64 import standard_b64encode
+    buf = io.StringIO()
+    kw['file'] = buf
     print(*a, **kw)
-    fd.flush()
+    text = buf.getvalue()
+    text = b'\x1bP@kitty-print|' + standard_b64encode(text.encode('utf-8')) + b'\x1b\\'
+    fobj = getattr(debug, 'fobj', sys.stdout.buffer)
+    fobj.write(text)
+    if hasattr(fobj, 'flush'):
+        fobj.flush()
 
 
 def write_all(fd, data):
@@ -52,6 +59,7 @@ class TermManager:
         self.input_fd = input_fd
         self.output_fd = output_fd
         self.original_fl = fcntl.fcntl(self.input_fd, fcntl.F_GETFL)
+        self.extra_finalize = None
         self.isatty = os.isatty(self.input_fd)
         if self.isatty:
             self.original_termios = termios.tcgetattr(self.input_fd)
@@ -66,6 +74,8 @@ class TermManager:
         if self.isatty:
             termios.tcsetattr(self.input_fd, termios.TCSADRAIN, self.original_termios)
         fcntl.fcntl(self.input_fd, fcntl.F_SETFL, self.original_fl)
+        if self.extra_finalize:
+            write_all(self.output_fd, self.extra_finalize)
         write_all(self.output_fd, reset_state())
 
     @contextmanager
@@ -120,9 +130,11 @@ class UnhandledException(Handler):
     def __init__(self, tb):
         self.tb = tb
 
-    def initialize(self, screen_size, quit_loop, wakeup):
-        Handler.initialize(self, screen_size, quit_loop, wakeup)
-        self.write(clear_screen())
+    def initialize(self):
+        self.cmd.clear_screen()
+        self.cmd.set_scrolling_region()
+        self.cmd.set_cursor_visible(True)
+        self.cmd.set_default_colors()
         self.write(self.tb.replace('\n', '\r\n'))
         self.write('\r\n')
         self.write('Press the Enter key to quit')
@@ -167,6 +179,24 @@ class Loop:
         self.sanitize_bracketed_paste = bool(sanitize_bracketed_paste)
         if self.sanitize_bracketed_paste:
             self.sanitize_ibp_pat = re.compile(sanitize_bracketed_paste)
+        self.jobs_queue = Queue()
+
+    def start_job(self, job_id, func, *args, **kw):
+        from threading import Thread
+        t = Thread(target=partial(self._run_job, job_id, func), args=args, kwargs=kw, name='LoopJob')
+        t.daemon = True
+        t.start()
+
+    def _run_job(self, job_id, func, *args, **kw):
+        try:
+            result = func(*args, **kw)
+        except Exception as err:
+            import traceback
+            entry = {'id': job_id, 'exception': err, 'tb': traceback.format_exc()}
+        else:
+            entry = {'id': job_id, 'result': result}
+        self.jobs_queue.put(entry)
+        self._wakeup_write(b'j')
 
     def _read_ready(self, handler):
         if not self.read_allowed:
@@ -210,7 +240,9 @@ class Loop:
                 self.handler.on_text(chunk, self.in_bracketed_paste)
 
     def _on_dcs(self, dcs):
-        pass
+        if dcs.startswith('@kitty-cmd'):
+            import json
+            self.handler.on_kitty_cmd_response(json.loads(dcs[len('@kitty-cmd'):]))
 
     def _on_csi(self, csi):
         q = csi[-1]
@@ -250,6 +282,9 @@ class Loop:
                         self.handler.on_eot()
                         return
                 self.handler.on_key(k)
+        elif apc.startswith('G'):
+            if self.handler.image_manager is not None:
+                self.handler.image_manager.handle_response(apc)
 
     def _write_ready(self, handler):
         if len(handler.write_buf) > self.iov_limit:
@@ -283,6 +318,17 @@ class Loop:
             handler.on_term()
         if b'i' in data:
             handler.on_interrupt()
+        if b'j' in data:
+            while True:
+                try:
+                    entry = self.jobs_queue.get_nowait()
+                except Empty:
+                    break
+                else:
+                    job_id = entry.pop('id')
+                    handler.on_job_done(job_id, entry)
+        if b'1' in data:
+            handler.on_wakeup()
 
     def _wakeup_write(self, val):
         while not os.write(self.wakeup_write_fd, val):
@@ -327,31 +373,31 @@ class Loop:
             signal.signal(signal.SIGINT, self._on_sigint)
             handler.write_buf = []
             handler._term_manager = term_manager
+            image_manager = None
+            if handler.image_manager_class is not None:
+                image_manager = handler.image_manager_class(handler)
             keep_going = True
             try:
-                handler.initialize(screen_size(), self.quit, self.wakeup)
+                handler._initialize(screen_size(), self.quit, self.wakeup, self.start_job, debug, image_manager)
+                with handler:
+                    while keep_going:
+                        has_data_to_write = bool(handler.write_buf)
+                        if not has_data_to_write and not self.read_allowed:
+                            break
+                        if has_data_to_write != waiting_for_write:
+                            waiting_for_write = has_data_to_write
+                            self._modify_output_selector(waiting_for_write)
+                        events = select()
+                        for key, mask in events:
+                            key.data(handler)
             except Exception:
                 import traceback
                 tb = traceback.format_exc()
                 self.return_code = 1
                 keep_going = False
-            while keep_going:
-                has_data_to_write = bool(handler.write_buf)
-                if not has_data_to_write and not self.read_allowed:
-                    break
-                if has_data_to_write != waiting_for_write:
-                    waiting_for_write = has_data_to_write
-                    self._modify_output_selector(waiting_for_write)
-                events = select()
-                for key, mask in events:
-                    try:
-                        key.data(handler)
-                    except Exception:
-                        import traceback
-                        tb = traceback.format_exc()
-                        self.return_code = 1
-                        keep_going = False
-                        break
+
+            term_manager.extra_finalize = b''.join(handler.write_buf).decode('utf-8')
+
             if tb is not None:
                 self._report_error_loop(tb, term_manager)
 
@@ -361,14 +407,15 @@ class Loop:
         handler = UnhandledException(tb)
         handler.write_buf = []
         handler._term_manager = term_manager
-        handler.initialize(screen_size(), self.quit, self.wakeup)
-        while True:
-            has_data_to_write = bool(handler.write_buf)
-            if not has_data_to_write and not self.read_allowed:
-                break
-            if has_data_to_write != waiting_for_write:
-                waiting_for_write = has_data_to_write
-                self._modify_output_selector(waiting_for_write)
-            events = select()
-            for key, mask in events:
-                key.data(handler)
+        handler._initialize(screen_size(), self.quit, self.wakeup, self.start_job, debug)
+        with handler:
+            while True:
+                has_data_to_write = bool(handler.write_buf)
+                if not has_data_to_write and not self.read_allowed:
+                    break
+                if has_data_to_write != waiting_for_write:
+                    waiting_for_write = has_data_to_write
+                    self._modify_output_selector(waiting_for_write)
+                events = select()
+                for key, mask in events:
+                    key.data(handler)

kittens/tui/operations.py

@@ -4,8 +4,9 @@
 
 import sys
 from contextlib import contextmanager
+from functools import wraps
 
-from kitty.terminfo import string_capabilities
+from kitty.rgb import Color, color_as_sharp, to_color
 
 S7C1T = '\033 F'
 SAVE_CURSOR = '\0337'
@@ -35,34 +36,69 @@ MODES = dict(
 )
 
 
-def set_mode(which, private=True):
+def set_mode(which, private=True) -> str:
     num, private = MODES[which]
     return '\033[{}{}h'.format(private, num)
 
 
-def reset_mode(which):
+def reset_mode(which) -> str:
     num, private = MODES[which]
     return '\033[{}{}l'.format(private, num)
 
 
-def clear_screen():
-    return string_capabilities['clear'].replace(r'\E', '\033')
+def clear_screen() -> str:
+    return '\033[H\033[2J'
 
 
-def set_window_title(value):
+def clear_to_eol() -> str:
+    return '\033[K'
+
+
+def bell() -> str:
+    return '\a'
+
+
+def beep() -> str:
+    return '\a'
+
+
+def set_window_title(value) -> str:
     return ('\033]2;' + value.replace('\033', '').replace('\x9c', '') + '\033\\')
 
 
-def set_line_wrapping(yes_or_no):
+def set_line_wrapping(yes_or_no) -> str:
     return (set_mode if yes_or_no else reset_mode)('DECAWM')
 
 
-def set_cursor_visible(yes_or_no):
+def set_cursor_visible(yes_or_no) -> str:
     return (set_mode if yes_or_no else reset_mode)('DECTCEM')
 
 
+def set_cursor_position(x, y) -> str:  # (0, 0) is top left
+    return '\033[{};{}H'.format(y + 1, x + 1)
+
+
+def set_scrolling_region(screen_size=None, top=None, bottom=None) -> str:
+    if screen_size is None:
+        return '\033[r'
+    if top is None:
+        top = 0
+    if bottom is None:
+        bottom = screen_size.rows - 1
+    if bottom < 0:
+        bottom = screen_size.rows - 1 + bottom
+    else:
+        bottom += 1
+    return '\033[{};{}r'.format(top + 1, bottom + 1)
+
+
+def scroll_screen(amt=1) -> str:
+    return '\033[' + str(abs(amt)) + ('T' if amt < 0 else 'S')
+
+
 STANDARD_COLORS = {name: i for i, name in enumerate(
     'black red green yellow blue magenta cyan gray'.split())}
+STANDARD_COLORS['white'] = STANDARD_COLORS['gray']
 UNDERLINE_STYLES = {name: i + 1 for i, name in enumerate(
     'straight double curly'.split())}
 
@@ -77,20 +113,20 @@ def color_code(color, intense=False, base=30):
     return e
 
 
-def sgr(*parts):
+def sgr(*parts) -> str:
     return '\033[{}m'.format(';'.join(parts))
 
 
-def colored(text, color, intense=False, reset_to=None, reset_to_intense=False):
+def colored(text, color, intense=False, reset_to=None, reset_to_intense=False) -> str:
     e = color_code(color, intense)
     return '\033[{}m{}\033[{}m'.format(e, text, 39 if reset_to is None else color_code(reset_to, reset_to_intense))
 
 
-def faint(text):
+def faint(text) -> str:
     return colored(text, 'black', True)
 
 
-def styled(text, fg=None, bg=None, fg_intense=False, bg_intense=False, italic=None, bold=None, underline=None, underline_color=None, reverse=None):
+def styled(text, fg=None, bg=None, fg_intense=False, bg_intense=False, italic=None, bold=None, underline=None, underline_color=None, reverse=None) -> str:
     start, end = [], []
     if fg is not None:
         start.append(color_code(fg, fg_intense))
@@ -120,6 +156,26 @@ def styled(text, fg=None, bg=None, fg_intense=False, bg_intense=False, italic=No
     return '\033[{}m{}\033[{}m'.format(';'.join(start), text, ';'.join(end))
 
 
+def serialize_gr_command(cmd, payload=None):
+    cmd = ','.join('{}={}'.format(k, v) for k, v in cmd.items())
+    ans = []
+    w = ans.append
+    w(b'\033_G'), w(cmd.encode('ascii'))
+    if payload:
+        w(b';')
+        w(payload)
+    w(b'\033\\')
+    return b''.join(ans)
+
+
+def gr_command(cmd, payload=None) -> str:
+    return serialize_gr_command(cmd, payload)
+
+
+def clear_images_on_screen(delete_data=False) -> str:
+    return serialize_gr_command({'a': 'd', 'd': 'A' if delete_data else 'a'})
+
+
 def init_state(alternate_screen=True):
     ans = (
         S7C1T + SAVE_CURSOR + SAVE_PRIVATE_MODE_VALUES + reset_mode('LNM') +
@@ -160,3 +216,33 @@ def alternate_screen(f=None):
     print(set_mode('ALTERNATE_SCREEN'), end='', file=f)
     yield
     print(reset_mode('ALTERNATE_SCREEN'), end='', file=f)
+
+
+def set_default_colors(fg=None, bg=None) -> str:
+    ans = ''
+    if fg is None:
+        ans += '\x1b]110\x1b\\'
+    else:
+        ans += '\x1b]10;{}\x1b\\'.format(color_as_sharp(fg if isinstance(fg, Color) else to_color(fg)))
+    if bg is None:
+        ans += '\x1b]111\x1b\\'
+    else:
+        ans += '\x1b]11;{}\x1b\\'.format(color_as_sharp(bg if isinstance(bg, Color) else to_color(bg)))
+    return ans
+
+
+all_cmds = tuple(
+        (name, obj) for name, obj in globals().items()
+        if hasattr(obj, '__annotations__') and obj.__annotations__.get('return') is str)
+
+
+def writer(handler, func):
+    @wraps(func)
+    def f(self, *a, **kw):
+        handler.write(func(*a, **kw))
+    return f
+
+
+def commander(handler):
+    ans = {name: writer(handler, obj) for name, obj in all_cmds}
+    return type('CMD', (), ans)()

kittens/unicode_input/main.py

@@ -5,7 +5,6 @@
 import os
 import string
 import subprocess
-import sys
 from functools import lru_cache
 from gettext import gettext as _
 
@@ -43,7 +42,7 @@ def codepoint_ok(code):
 @lru_cache(maxsize=256)
 def points_for_word(w):
     from .unicode_names import codepoints_for_word
-    return codepoints_for_word(w)
+    return codepoints_for_word(w.lower())
 
 
 @lru_cache(maxsize=4096)
@@ -57,7 +56,7 @@ def name(cp):
 @lru_cache(maxsize=256)
 def codepoints_matching_search(parts):
     ans = []
-    if parts and parts[0]:
+    if parts and parts[0] and len(parts[0]) > 1:
         codepoints = points_for_word(parts[0])
         for word in parts[1:]:
             pts = points_for_word(word)
@@ -223,6 +222,8 @@ class Table:
         return self.text
 
     def move_current(self, rows=0, cols=0):
+        if len(self.codepoints) == 0:
+            return
         if cols:
             self.current_idx = (self.current_idx + len(self.codepoints) + cols) % len(self.codepoints)
             self.layout_dirty = True
@@ -326,8 +327,7 @@ class UnicodeInput(Handler):
         self.write(set_line_wrapping(False))
         self.write(set_window_title(_('Unicode input')))
 
-    def initialize(self, *args):
-        Handler.initialize(self, *args)
+    def initialize(self):
         self.init_terminal_state()
         self.draw_screen()
 
@@ -452,26 +452,34 @@ class UnicodeInput(Handler):
         self.refresh()
 
 
-def run_loop(args):
+def main(args):
     loop = Loop()
     with cached_values_for('unicode-input') as cached_values:
         handler = UnicodeInput(cached_values)
         loop.loop(handler)
         if handler.current_char and loop.return_code == 0:
-            print('OK:', hex(ord(handler.current_char))[2:])
             try:
                 handler.recent.remove(ord(handler.current_char))
             except Exception:
                 pass
             recent = [ord(handler.current_char)] + handler.recent
             cached_values['recent'] = recent[:len(DEFAULT_SET)]
-    return loop.return_code
+            return handler.current_char
+    if loop.return_code != 0:
+        raise SystemExit(loop.return_code)
 
 
-def main(args=sys.argv):
-    try:
-        raise SystemExit(run_loop(args))
-    except Exception:
-        import traceback
-        traceback.print_exc()
-        input(_('Press Enter to quit.'))
+def handle_result(args, current_char, target_window_id, boss):
+    w = boss.window_id_map.get(target_window_id)
+    if w is not None:
+        w.paste(current_char)
+
+
+if __name__ == '__main__':
+    import sys
+    if '-h' in sys.argv or '--help' in sys.argv:
+        print('Choose a unicode character to input into the terminal')
+        raise SystemExit(0)
+    ans = main(sys.argv)
+    if ans:
+        print(ans)

kittens/unicode_input/unicode_names.c

@@ -9,32 +9,61 @@
 
 static PyObject*
 all_words(PYNOARG) {
-    PyObject *ans = PyTuple_New(arraysz(idx_to_word));
+    PyObject *ans = PyTuple_New(arraysz(all_words_map));
     if (!ans) return NULL;
-    for (size_t i = 0; i < arraysz(idx_to_word); i++) {
-        PyObject *w = PyUnicode_FromString(idx_to_word[i]);
+    for (size_t i = 0; i < arraysz(all_words_map); i++) {
+        PyObject *w = PyUnicode_FromString(all_words_map[i]);
         if (w == NULL) { Py_DECREF(ans); return NULL; }
         PyTuple_SET_ITEM(ans, i, w);
     }
     return ans;
 }
 
+static inline void
+add_matches(const word_trie *wt, char_type *codepoints, size_t *pos, const size_t sz) {
+    size_t num = mark_groups[wt->match_offset];
+    for (size_t i = wt->match_offset + 1; i < wt->match_offset + 1 + num && *pos < sz; i++, (*pos)++) {
+        codepoints[*pos] = mark_to_cp[mark_groups[i]];
+    }
+}
+
+static void
+process_trie_node(const word_trie *wt, char_type *codepoints, size_t *pos, const size_t sz) {
+    if (wt->match_offset) add_matches(wt, codepoints, pos, sz);
+    size_t num_children = children_array[wt->children_offset];
+    if (!num_children) return;
+    for (size_t c = wt->children_offset + 1; c < wt->children_offset + 1 + num_children; c++) {
+        if (*pos > sz) return;
+        uint32_t x = children_array[c];
+        process_trie_node(&all_trie_nodes[x >> 8], codepoints, pos, sz);
+    }
+}
+
 static inline PyObject*
 codepoints_for_word(const char *word, size_t len) {
-    PyObject *ans = PyFrozenSet_New(NULL); if (ans == NULL) return NULL;
-    const unsigned short *words = words_for_first_letter[(unsigned)*word];
-    if (words == NULL) return ans;
-    for (unsigned short i = 1; i <= words[0]; i++) {
-        unsigned short word_idx = words[i];
-        const char *w = idx_to_word[word_idx];
-        if (strncmp(word, w, len) == 0 && strlen(w) == len) {
-            const char_type* codepoints = codepoints_for_word_idx[word_idx];
-            for (char_type i = 1; i <= codepoints[0]; i++) {
-                PyObject *t = PyLong_FromUnsignedLong(codepoints[i]); if (t == NULL) { Py_DECREF(ans); return NULL; }
-                int ret = PySet_Add(ans, t); Py_DECREF(t); if (ret != 0) { Py_DECREF(ans); return NULL; }
+    const word_trie *wt = all_trie_nodes;
+    for (size_t i = 0; i < len; i++) {
+        unsigned char ch = word[i];
+        size_t num_children = children_array[wt->children_offset];
+        if (!num_children) return PyFrozenSet_New(NULL);
+        bool found = false;
+        for (size_t c = wt->children_offset + 1; c < wt->children_offset + 1 + num_children; c++) {
+            uint32_t x = children_array[c];
+            if ((x & 0xff) == ch) {
+                found = true;
+                wt = &all_trie_nodes[x >> 8];
+                break;
             }
-            break;
         }
+        if (!found) return PyFrozenSet_New(NULL);
+    }
+    static char_type codepoints[1024];
+    size_t cpos = 0;
+    process_trie_node(wt, codepoints, &cpos, arraysz(codepoints));
+    PyObject *ans = PyFrozenSet_New(NULL); if (ans == NULL) return NULL;
+    for (size_t i = 0; i < cpos; i++) {
+        PyObject *t = PyLong_FromUnsignedLong(codepoints[i]); if (t == NULL) { Py_DECREF(ans); return NULL; }
+        int ret = PySet_Add(ans, t); Py_DECREF(t); if (ret != 0) { Py_DECREF(ans); return NULL; }
     }
     return ans;
 }
@@ -56,12 +85,13 @@ nfc(PyObject *self UNUSED, PyObject *args) {
 }
 
 static PyMethodDef module_methods[] = {
-    METHODB(all_words, METH_NOARGS),
+    {"all_words", (PyCFunction)all_words, METH_NOARGS, ""},
     {"codepoints_for_word", (PyCFunction)cfw, METH_VARARGS, ""},
     {"name_for_codepoint", (PyCFunction)nfc, METH_VARARGS, ""},
     {NULL, NULL, 0, NULL}        /* Sentinel */
 };
 
+#if PY_VERSION_HEX >= 0x03000000
 static struct PyModuleDef module = {
    .m_base = PyModuleDef_HEAD_INIT,
    .m_name = "unicode_names",   /* name of module */
@@ -79,3 +109,13 @@ PyInit_unicode_names(void) {
     if (m == NULL) return NULL;
     return m;
 }
+#else
+EXPORTED
+initunicode_names(void) {
+    PyObject *m;
+    m = Py_InitModule3("unicode_names", module_methods,
+    ""
+    );
+    if (m == NULL) return;
+}
+#endif

kittens/url_hints/main.py

@@ -1,247 +0,0 @@
-#!/usr/bin/env python
-# vim:fileencoding=utf-8
-# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
-
-import re
-import string
-import subprocess
-import sys
-from collections import namedtuple
-from functools import lru_cache, partial
-from gettext import gettext as _
-
-from kitty.cli import parse_args
-from kitty.key_encoding import ESCAPE, backspace_key, enter_key
-from kitty.utils import command_for_open
-
-from ..tui.handler import Handler
-from ..tui.loop import Loop
-from ..tui.operations import (
-    clear_screen, faint, set_cursor_visible, set_window_title, styled
-)
-
-Mark = namedtuple('Mark', 'index start end text')
-URL_PREFIXES = 'http https file ftp'.split()
-HINT_ALPHABET = string.digits + string.ascii_lowercase
-
-
-@lru_cache(maxsize=2048)
-def encode_hint(num):
-    res = ''
-    d = len(HINT_ALPHABET)
-    while not res or num > 0:
-        num, i = divmod(num, d)
-        res = HINT_ALPHABET[i] + res
-    return res
-
-
-def decode_hint(x):
-    return int(x, 36)
-
-
-def render(lines, current_input):
-    ans = []
-
-    def mark(m):
-        hint = encode_hint(m.index)
-        text = m.text
-        if current_input and not hint.startswith(current_input):
-            return faint(text)
-        hint = hint[len(current_input):] or ' '
-        text = text[len(hint):]
-        return styled(
-            hint,
-            fg='black',
-            bg='green',
-            bold=True
-        ) + styled(
-            text, fg='gray', fg_intense=True, bold=True
-        )
-
-    for line, marks in lines:
-        if not marks:
-            ans.append(faint(line))
-            continue
-        buf = []
-
-        for i, m in enumerate(marks):
-            if i == 0 and m.start:
-                buf.append(faint(line[:m.start]))
-            buf.append(mark(m))
-            if m is not marks[-1]:
-                buf.append(faint(line[m.end:marks[i + 1].start]))
-
-        rest = line[marks[-1].end:]
-        if rest:
-            buf.append(faint(rest))
-
-        ans.append(''.join(buf))
-    return '\r\n'.join(ans)
-
-
-class URLHints(Handler):
-
-    def __init__(self, lines, index_map):
-        self.lines, self.index_map = tuple(lines), index_map
-        self.current_input = ''
-        self.current_text = None
-        self.chosen = None
-
-    def init_terminal_state(self):
-        self.write(set_cursor_visible(False))
-        self.write(set_window_title(_('Choose URL')))
-
-    def initialize(self, *args):
-        Handler.initialize(self, *args)
-        self.init_terminal_state()
-        self.draw_screen()
-
-    def on_text(self, text, in_bracketed_paste):
-        changed = False
-        for c in text:
-            if c in HINT_ALPHABET:
-                self.current_input += c
-                changed = True
-        if changed:
-            matches = [
-                t for idx, t in self.index_map.items()
-                if encode_hint(idx).startswith(self.current_input)
-            ]
-            if len(matches) == 1:
-                self.chosen = matches[0]
-                self.quit_loop(0)
-                return
-            self.current_text = None
-            self.draw_screen()
-
-    def on_key(self, key_event):
-        if key_event is backspace_key:
-            self.current_input = self.current_input[:-1]
-            self.current_text = None
-            self.draw_screen()
-        elif key_event is enter_key and self.current_input:
-            idx = decode_hint(self.current_input)
-            self.chosen = self.index_map[idx]
-            self.quit_loop(0)
-        elif key_event.key is ESCAPE:
-            self.quit_loop(1)
-
-    def on_interrupt(self):
-        self.quit_loop(1)
-
-    def on_eot(self):
-        self.quit_loop(1)
-
-    def on_resize(self, new_size):
-        Handler.on_resize(self, new_size)
-        self.draw_screen()
-
-    def draw_screen(self):
-        if self.current_text is None:
-            self.current_text = render(self.lines, self.current_input)
-        self.write(clear_screen())
-        self.write(self.current_text)
-
-
-def regex_finditer(pat, line):
-    for m in pat.finditer(line):
-        s, e = m.span()
-        if e - s > 2:
-            yield s, e
-
-
-def find_urls(pat, line):
-    for m in pat.finditer(line):
-        s, e = m.span()
-        if s > 4 and line[s - 5:s] == 'link:':  # asciidoc URLs
-            url = line[s:e]
-            idx = url.rfind('[')
-            if idx > -1:
-                e -= len(url) - idx
-        yield s, e
-
-
-def mark(finditer, line, index_map):
-    marks = []
-    for s, e in finditer(line):
-        idx = len(index_map)
-        text = line[s:e]
-        marks.append(Mark(idx, s, e, text))
-        index_map[idx] = text
-    return line, marks
-
-
-def run_loop(args, lines, index_map):
-    loop = Loop()
-    handler = URLHints(lines, index_map)
-    loop.loop(handler)
-    if handler.chosen and loop.return_code == 0:
-        cmd = command_for_open(args.program)
-        ret = subprocess.Popen(cmd + [handler.chosen]).wait()
-        if ret != 0:
-            print('URL handler "{}" failed with return code: {}'.format(' '.join(cmd), ret), file=sys.stderr)
-            input('Press Enter to quit')
-            loop.return_code = ret
-    raise SystemExit(loop.return_code)
-
-
-def run(args, source_file=None):
-    if source_file is None:
-        text = sys.stdin.buffer.read().decode('utf-8')
-        sys.stdin = open('/dev/tty')
-    else:
-        with open(source_file, 'r') as f:
-            text = f.read()
-    if args.regex is None:
-        from .url_regex import url_delimiters
-        url_pat = '(?:{})://[^{}]{{3,}}'.format(
-            '|'.join(args.url_prefixes.split(',')), url_delimiters
-        )
-        finditer = partial(find_urls, re.compile(url_pat))
-    else:
-        finditer = partial(regex_finditer, re.compile(args.regex))
-    lines = []
-    index_map = {}
-    for line in text.splitlines():
-        marked = mark(finditer, line, index_map)
-        lines.append(marked)
-    if not index_map:
-        input(_('No URLs found, press Enter to abort.'))
-        return
-
-    try:
-        run_loop(args, lines, index_map)
-    except Exception:
-        import traceback
-        traceback.print_exc()
-        input(_('Press Enter to quit'))
-
-
-OPTIONS = partial('''\
---program
-default=default
-What program to use to open matched URLs. Defaults
-to the default URL open program for the operating system.
-
-
---regex
-Instead of searching for URLs search for the specified regular
-expression instead.
-
-
---url-prefixes
-default={0}
-Comma separated list of recognized URL prefixes. Defaults to:
-{0}
-'''.format, ','.join(sorted(URL_PREFIXES)))
-
-
-def main(args=sys.argv):
-    msg = 'Highlight URLs inside the specified text'
-    try:
-        args, items = parse_args(args[1:], OPTIONS, '[path to file or omit to use stdin]', msg, 'url_hints')
-    except SystemExit as e:
-        print(e.args[0], file=sys.stderr)
-        input('Press enter to quit...')
-        return 1
-    run(args, (items or [None])[0])

kittens/url_hints/url_regex.py

@@ -1 +0,0 @@
-url_delimiters = '\x00-\x20\x7f-\xa0\xad\u0600-\u0605\u061c\u06dd\u070f\u08e2\u1680\u180e\u2000-\u200f\u2028-\u202f\u205f-\u2064\u2066-\u206f\u3000\ud800-\uf8ff\ufeff\ufff9-\ufffb\U000110bd\U0001bca0-\U0001bca3\U0001d173-\U0001d17a\U000e0001\U000e0020-\U000e007f\U000f0000-\U000ffffd\U00100000-\U0010fffd'  # noqa

kitty/border_vertex.glsl

@@ -3,6 +3,7 @@ uniform uvec2 viewport;
 uniform vec3 default_bg;
 uniform vec3 active_border_color;
 uniform vec3 inactive_border_color;
+uniform vec3 bell_border_color;
 in uvec4 rect;  // left, top, right, bottom
 in uint rect_color;
 out vec3 color;
@@ -37,5 +38,5 @@ void main() {
     gl_Position = vec4(to_opengl(rect[pos.x], rect[pos.y]), 0, 1);
     int rc = int(rect_color);
     vec3 window_bg = vec3(to_color(rect_color >> 24), to_color(rect_color >> 16), to_color(rect_color >> 8));
-    color = float(1 & rc) * default_bg + float((2 & rc) >> 1) * active_border_color + float((4 & rc) >> 2) * inactive_border_color + float((8 & rc) >> 3) * window_bg;
+    color = float(1 & rc) * default_bg + float((2 & rc) >> 1) * active_border_color + float((4 & rc) >> 2) * inactive_border_color + float((8 & rc) >> 3) * window_bg + float((16 & rc) >> 4) * bell_border_color;
 }

kitty/borders.py

@@ -64,7 +64,7 @@ class Borders:
                 g = w.geometry
                 if bw > 0 and draw_window_borders:
                     # Draw the border rectangles
-                    color = 2 if w is active_window else 4
+                    color = 2 if w is active_window else (16 if w.needs_attention else 4)
                     border(
                         self.os_window_id, self.tab_id,
                         color, bw, g.left - fw, g.top - fw, g.right + fw,

kitty/boss.py

@@ -4,8 +4,8 @@
 
 import atexit
 import json
+import os
 import re
-import socket
 from functools import partial
 from gettext import gettext as _
 from weakref import WeakValueDictionary
@@ -14,17 +14,21 @@ from .cli import create_opts, parse_args
 from .config import (
     MINIMUM_FONT_SIZE, initial_window_size, prepare_config_file_for_editing
 )
-from .constants import appname, editor, set_boss
+from .config_utils import to_cmdline
+from .constants import (
+    appname, config_dir, editor, set_boss, supports_primary_selection
+)
 from .fast_data_types import (
     ChildMonitor, create_os_window, current_os_window, destroy_global_data,
-    destroy_sprite_map, get_clipboard_string, glfw_post_empty_event,
-    layout_sprite_map, mark_os_window_for_close, set_clipboard_string,
-    set_dpi_from_os_window, show_window, toggle_fullscreen,
-    viewport_for_window
+    destroy_sprite_map, get_clipboard_string,
+    glfw_post_empty_event, layout_sprite_map, mark_os_window_for_close,
+    set_clipboard_string, set_dpi_from_os_window, set_in_sequence_mode,
+    show_window, toggle_fullscreen, viewport_for_window
 )
 from .fonts.render import prerender, resize_fonts, set_font_family
-from .keys import get_shortcut
+from .keys import get_shortcut, shortcut_matches
 from .remote_control import handle_cmd
+from .rgb import Color, color_from_int
 from .session import create_session
 from .tabs import SpecialWindow, SpecialWindowInstance, TabManager
 from .utils import (
@@ -40,6 +44,7 @@ def initialize_renderer():
 
 
 def listen_on(spec):
+    import socket
     family, address, socket_path = parse_address_spec(spec)
     s = socket.socket(family)
     atexit.register(remove_socket_file, s, socket_path)
@@ -79,6 +84,8 @@ class Boss:
 
     def __init__(self, os_window_id, opts, args, cached_values):
         self.window_id_map = WeakValueDictionary()
+        self.startup_colors = {k: opts[k] for k in opts if isinstance(opts[k], Color)}
+        self.pending_sequences = None
         self.cached_values = cached_values
         self.os_window_map = {}
         self.cursor_blinking = True
@@ -116,6 +123,7 @@ class Boss:
         self.os_window_map[os_window_id] = tm
         if dpi_changed:
             self.on_dpi_change(os_window_id)
+        return os_window_id
 
     def list_os_windows(self):
         for os_window_id, tm in self.os_window_map.items():
@@ -124,24 +132,45 @@ class Boss:
                 'tabs': list(tm.list_tabs()),
             }
 
+    @property
+    def all_tab_managers(self):
+        yield from self.os_window_map.values()
+
+    @property
+    def all_tabs(self):
+        for tm in self.all_tab_managers:
+            yield from tm
+
+    @property
+    def all_windows(self):
+        for tab in self.all_tabs:
+            yield from tab
+
     def match_windows(self, match):
         try:
             field, exp = match.split(':', 1)
         except ValueError:
             return
-        pat = re.compile(exp)
-        for tm in self.os_window_map.values():
-            for tab in tm:
-                for window in tab:
-                    if window.matches(field, pat):
-                        yield window
+        if field == 'num':
+            tab = self.active_tab
+            if tab is not None:
+                try:
+                    w = tab.get_nth_window(int(exp))
+                except Exception:
+                    return
+                if w is not None:
+                    yield w
+        else:
+            pat = re.compile(exp)
+            for window in self.all_windows:
+                if window.matches(field, pat):
+                    yield window
 
     def tab_for_window(self, window):
-        for tm in self.os_window_map.values():
-            for tab in tm:
-                for w in tab:
-                    if w.id == window.id:
-                        return tab
+        for tab in self.all_tabs:
+            for w in tab:
+                if w.id == window.id:
+                    return tab
 
     def match_tabs(self, match):
         try:
@@ -149,14 +178,12 @@ class Boss:
         except ValueError:
             return
         pat = re.compile(exp)
-        tms = tuple(self.os_window_map.values())
         found = False
         if field in ('title', 'id'):
-            for tm in tms:
-                for tab in tm:
-                    if tab.matches(field, pat):
-                        yield tab
-                        found = True
+            for tab in self.all_tabs:
+                if tab.matches(field, pat):
+                    yield tab
+                    found = True
         if not found:
             tabs = {self.tab_for_window(w) for w in self.match_windows(match)}
             for tab in tabs:
@@ -176,7 +203,7 @@ class Boss:
     def _new_os_window(self, args, cwd_from=None):
         sw = self.args_to_special_window(args, cwd_from) if args else None
         startup_session = create_session(self.opts, special_window=sw, cwd_from=cwd_from)
-        self.add_os_window(startup_session)
+        return self.add_os_window(startup_session)
 
     def new_os_window(self, *args):
         self._new_os_window(args)
@@ -192,7 +219,7 @@ class Boss:
 
     def _handle_remote_command(self, cmd, window=None):
         response = None
-        if self.opts.allow_remote_control:
+        if self.opts.allow_remote_control or getattr(window, 'allow_remote_control', False):
             try:
                 response = handle_cmd(self, window, cmd)
             except Exception as err:
@@ -220,7 +247,9 @@ class Boss:
                 args, rest = parse_args(msg['args'][1:])
                 args.args = rest
                 opts = create_opts(args)
-                session = create_session(opts, args)
+                if not os.path.isabs(args.directory):
+                    args.directory = os.path.join(msg['cwd'], args.directory)
+                session = create_session(opts, args, respect_cwd=True)
                 self.add_os_window(session, wclass=args.cls, wname=args.name, size=initial_window_size(opts, self.cached_values), startup_id=startup_id)
             else:
                 log_error('Unknown message received from peer, ignoring')
@@ -352,8 +381,56 @@ class Boss:
     def dispatch_special_key(self, key, scancode, action, mods):
         # Handles shortcuts, return True if the key was consumed
         key_action = get_shortcut(self.opts.keymap, mods, key, scancode)
-        self.current_key_press_info = key, scancode, action, mods
-        return self.dispatch_action(key_action)
+        if key_action is None:
+            sequences = get_shortcut(self.opts.sequence_map, mods, key, scancode)
+            if sequences:
+                self.pending_sequences = sequences
+                set_in_sequence_mode(True)
+                return True
+        else:
+            self.current_key_press_info = key, scancode, action, mods
+            return self.dispatch_action(key_action)
+
+    def process_sequence(self, key, scancode, action, mods):
+        if not self.pending_sequences:
+            set_in_sequence_mode(False)
+
+        remaining = {}
+        matched_action = None
+        for seq, key_action in self.pending_sequences.items():
+            if shortcut_matches(seq[0], mods, key, scancode):
+                seq = seq[1:]
+                if seq:
+                    remaining[seq] = key_action
+                else:
+                    matched_action = key_action
+
+        if remaining:
+            self.pending_sequences = remaining
+        else:
+            self.pending_sequences = None
+            set_in_sequence_mode(False)
+            if matched_action is not None:
+                self.dispatch_action(matched_action)
+
+    def start_resizing_window(self):
+        w = self.active_window
+        if w is None:
+            return
+        overlay_window = self._run_kitten('resize_window', args=[
+            '--horizontal-increment={}'.format(self.opts.window_resize_step_cells),
+            '--vertical-increment={}'.format(self.opts.window_resize_step_lines)
+        ])
+        if overlay_window is not None:
+            overlay_window.allow_remote_control = True
+
+    def resize_layout_window(self, window, increment, is_horizontal, reset=False):
+        tab = window.tabref()
+        if tab is None or not increment:
+            return False
+        if reset:
+            return tab.reset_window_sizes()
+        return tab.resize_window_by(window.id, increment, is_horizontal)
 
     def default_bg_changed_for(self, window_id):
         w = self.window_id_map.get(window_id)
@@ -395,6 +472,12 @@ class Boss:
             w = tm.active_window
             if w is not None:
                 w.focus_changed(focused)
+            tm.mark_tab_bar_dirty()
+
+    def update_tab_bar_data(self, os_window_id):
+        tm = self.os_window_map.get(os_window_id)
+        if tm is not None:
+            tm.update_tab_bar_data()
 
     def on_drop(self, os_window_id, paths):
         tm = self.os_window_map.get(os_window_id)
@@ -425,93 +508,111 @@ class Boss:
         cmd = ['kitty', '+runpy', 'import os, sys, time; time.sleep(0.05); os.execvp(sys.argv[1], sys.argv[1:])'] + editor + [confpath]
         self.new_os_window(*cmd)
 
-    def input_unicode_character(self):
-        w = self.active_window
-        tab = self.active_tab
-        if w is not None and tab is not None and w.overlay_for is None:
-            overlay_window = tab.new_special_window(
-                SpecialWindow(
-                    ['kitty', '+runpy', 'from kittens.unicode_input.main import main; main()'],
-                    overlay_for=w.id))
-            overlay_window.action_on_close = partial(self.send_unicode_character, w.id)
-
     def get_output(self, source_window, num_lines=1):
         output = ''
         s = source_window.screen
+        if num_lines is None:
+            num_lines = s.lines
         for i in range(min(num_lines, s.lines)):
             output += str(s.linebuf.line(i))
         return output
 
-    def send_unicode_character(self, target_window_id, source_window):
-        w = self.window_id_map.get(target_window_id)
-        if w is not None:
-            output = self.get_output(source_window)
-            if output.startswith('OK: '):
-                try:
-                    text = chr(int(output.partition(' ')[2], 16))
-                except Exception:
-                    import traceback
-                    traceback.print_exc()
-                else:
-                    w.paste(text)
-
-    def set_tab_title(self):
+    def _run_kitten(self, kitten, args=()):
         w = self.active_window
         tab = self.active_tab
         if w is not None and tab is not None and w.overlay_for is None:
-            args = ['--name=tab-title', '--message', _('Enter the new title for this tab below.')]
-            overlay_window = tab.new_special_window(
-                SpecialWindow(
-                    ['kitty', '+runpy', 'from kittens.ask.main import main; main()'] + args,
-                    overlay_for=w.id))
-            overlay_window.action_on_close = partial(self.do_set_tab_title, tab.id)
-
-    def do_set_tab_title(self, tab_id, source_window):
-        output = self.get_output(source_window)
-        if output.startswith('OK: '):
-            title = json.loads(output.partition(' ')[2].strip())
-            tm = self.active_tab_manager
-            if tm is not None and title:
-                for tab in tm.tabs:
-                    if tab.id == tab_id:
-                        tab.set_title(title)
-                        break
-
-    def run_simple_kitten(self, type_of_input, kitten, *args):
-        import shlex
-        w = self.active_window
-        tab = self.active_tab
-        if w is not None and tab is not None and w.overlay_for is None:
-            cmdline = args[0] if args else ''
-            args = shlex.split(cmdline) if cmdline else []
-            if '--program' not in cmdline:
-                args.extend(('--program', self.opts.open_url_with))
-            if type_of_input in ('text', 'history', 'ansi', 'ansi-history'):
-                data = w.as_text(as_ansi='ansi' in type_of_input, add_history='history' in type_of_input).encode('utf-8')
-            elif type_of_input == 'none':
+            orig_args, args = list(args), list(args)
+            from kittens.runner import create_kitten_handler
+            end_kitten = create_kitten_handler(kitten, orig_args)
+            args[0:0] = [config_dir, kitten]
+            type_of_input = end_kitten.type_of_input
+            if type_of_input in ('text', 'history', 'ansi', 'ansi-history', 'screen', 'screen-history', 'screen-ansi', 'screen-ansi-history'):
+                data = w.as_text(
+                        as_ansi='ansi' in type_of_input,
+                        add_history='history' in type_of_input,
+                        add_wrap_markers='screen' in type_of_input
+                ).encode('utf-8')
+            elif type_of_input is None:
                 data = None
             else:
                 raise ValueError('Unknown type_of_input: {}'.format(type_of_input))
-            tab.new_special_window(
+            copts = {k: self.opts[k] for k in ('select_by_word_characters', 'open_url_with')}
+            overlay_window = tab.new_special_window(
                 SpecialWindow(
-                    ['kitty', '+runpy', 'from kittens.{}.main import main; main()'.format(kitten)] + args,
+                    ['kitty', '+runpy', 'from kittens.runner import main; main()'] + args,
                     stdin=data,
+                    env={
+                        'KITTY_COMMON_OPTS': json.dumps(copts),
+                        'PYTHONWARNINGS': 'ignore',
+                        'OVERLAID_WINDOW_LINES': str(w.screen.lines),
+                        'OVERLAID_WINDOW_COLS': str(w.screen.columns),
+                    },
                     overlay_for=w.id))
+            overlay_window.action_on_close = partial(self.on_kitten_finish, w.id, end_kitten)
+            return overlay_window
+
+    def kitten(self, kitten, *args):
+        import shlex
+        cmdline = args[0] if args else ''
+        args = shlex.split(cmdline) if cmdline else []
+        self._run_kitten(kitten, args)
+
+    def on_kitten_finish(self, target_window_id, end_kitten, source_window):
+        output = self.get_output(source_window, num_lines=None)
+        from kittens.runner import deserialize
+        data = deserialize(output)
+        if data is not None:
+            end_kitten(data, target_window_id, self)
+
+    def input_unicode_character(self):
+        self._run_kitten('unicode_input')
+
+    def set_tab_title(self):
+        tab = self.active_tab
+        if tab:
+            args = ['--name=tab-title', '--message', _('Enter the new title for this tab below.'), 'do_set_tab_title', str(tab.id)]
+            self._run_kitten('ask', args)
+
+    def do_set_tab_title(self, title, tab_id):
+        tm = self.active_tab_manager
+        if tm is not None and title:
+            tab_id = int(tab_id)
+            for tab in tm.tabs:
+                if tab.id == tab_id:
+                    tab.set_title(title)
+                    break
+
+    def kitty_shell(self, window_type):
+        cmd = ['kitty', '@']
+        if window_type == 'tab':
+            window = self._new_tab(cmd).active_window
+        elif window_type == 'os_window':
+            os_window_id = self._new_os_window(cmd)
+            window = self.os_window_map[os_window_id].active_window
+        elif window_type == 'overlay':
+            w = self.active_window
+            tab = self.active_tab
+            if w is not None and tab is not None and w.overlay_for is None:
+                window = tab.new_special_window(SpecialWindow(cmd, overlay_for=w.id))
+            else:
+                window = None
+        else:
+            window = self._new_window(cmd)
+        if window is not None:
+            window.allow_remote_control = True
 
     def switch_focus_to(self, window_idx):
         tab = self.active_tab
         tab.set_active_window_idx(window_idx)
-        old_focus = tab.active_window
-        if not old_focus.destroyed:
-            old_focus.focus_changed(False)
-        tab.active_window.focus_changed(True)
 
-    def open_url(self, url):
+    def open_url(self, url, program=None, cwd=None):
         if url:
-            open_url(url, self.opts.open_url_with)
+            if isinstance(program, str):
+                program = to_cmdline(program)
+            open_url(url, program or self.opts.open_url_with, cwd=cwd)
 
-    def open_url_lines(self, lines):
-        self.open_url(''.join(lines))
+    def open_url_lines(self, lines, program=None):
+        self.open_url(''.join(lines), program)
 
     def destroy(self):
         self.shutting_down = True
@@ -534,7 +635,7 @@ class Boss:
         self.paste_to_active_window(text)
 
     def paste_from_selection(self):
-        text = get_primary_selection()
+        text = get_primary_selection() if supports_primary_selection else get_clipboard_string()
         self.paste_to_active_window(text)
 
     def set_primary_selection(self):
@@ -607,7 +708,7 @@ class Boss:
                 special_window = self.args_to_special_window(args, cwd_from=cwd_from)
         tm = self.active_tab_manager
         if tm is not None:
-            tm.new_tab(special_window=special_window, cwd_from=cwd_from)
+            return tm.new_tab(special_window=special_window, cwd_from=cwd_from)
 
     def new_tab(self, *args):
         self._new_tab(args)
@@ -621,9 +722,9 @@ class Boss:
         tab = self.active_tab
         if tab is not None:
             if args:
-                tab.new_special_window(self.args_to_special_window(args, cwd_from=cwd_from))
+                return tab.new_special_window(self.args_to_special_window(args, cwd_from=cwd_from))
             else:
-                tab.new_window(cwd_from=cwd_from)
+                return tab.new_window(cwd_from=cwd_from)
 
     def new_window(self, *args):
         self._new_window(args)
@@ -644,3 +745,11 @@ class Boss:
         tm = self.active_tab_manager
         if tm is not None:
             tm.move_tab(-1)
+
+    def patch_colors(self, spec, configured=False):
+        if configured:
+            for k, v in spec.items():
+                if hasattr(self.opts, k):
+                    setattr(self.opts, k, color_from_int(v))
+        for tm in self.all_tab_managers:
+            tm.tab_bar.patch_colors(spec)

kitty/cell_fragment.glsl

@@ -19,7 +19,7 @@ in float bg_alpha;
 
 #ifdef NEEDS_FOREGROUND
 uniform sampler2DArray sprites;
-uniform float inactive_text_alpha;
+in float effective_text_alpha;
 in vec3 sprite_pos;
 in vec3 underline_pos;
 in vec3 strike_pos;
@@ -61,7 +61,7 @@ vec4 calculate_foreground() {
     // Since strike and text are the same color, we simply add the alpha values
     float combined_alpha = min(text_alpha + strike_alpha, 1.0f);
     // Underline color might be different, so alpha blend
-    return alpha_blend(decoration_fg, underline_alpha * inactive_text_alpha, fg, combined_alpha * inactive_text_alpha);
+    return alpha_blend(decoration_fg, underline_alpha * effective_text_alpha, fg, combined_alpha * effective_text_alpha);
 }
 #endif
 

kitty/cell_vertex.glsl

@@ -1,6 +1,7 @@
 #version GLSL_VERSION
 #define WHICH_PROGRAM
 #define NOT_TRANSPARENT
+#define SHIFTS
 
 // Inputs {{{
 layout(std140) uniform CellRenderData {
@@ -45,12 +46,15 @@ out float bg_alpha;
 #endif
 
 #ifdef NEEDS_FOREGROUND
+uniform float inactive_text_alpha;
+uniform float dim_opacity;
 out vec3 sprite_pos;
 out vec3 underline_pos;
 out vec3 strike_pos;
 out vec3 foreground;
 out vec3 decoration_fg;
 out float colored_sprite;
+out float effective_text_alpha;
 #endif
 
 
@@ -63,9 +67,6 @@ const uint ONE = uint(1);
 const uint TWO = uint(2);
 const uint THREE = uint(3);
 const uint FOUR = uint(4);
-const uint DECORATION_MASK = uint(3);
-const uint STRIKE_MASK = uint(1);
-const uint REVERSE_MASK = uint(1);
 
 vec3 color_to_vec(uint c) {
     uint r, g, b;
@@ -134,7 +135,7 @@ void main() {
     // set cell color indices {{{
     uvec2 default_colors = uvec2(default_fg, default_bg);
     uint text_attrs = sprite_coords[3];
-    uint is_inverted = ((text_attrs >> 6) & REVERSE_MASK) + inverted;
+    uint is_inverted = ((text_attrs >> REVERSE_SHIFT) & ONE) + inverted;
     int fg_index = fg_index_map[is_inverted];
     int bg_index = 1 - fg_index;
     float cursor = is_cursor(c, r);
@@ -151,13 +152,15 @@ void main() {
     // Foreground
     uint resolved_fg = resolve_color(colors[fg_index], default_colors[fg_index]);
     foreground = color_to_vec(resolved_fg);
+    float has_dim = float((text_attrs >> DIM_SHIFT) & ONE);
+    effective_text_alpha = inactive_text_alpha * mix(1.0, dim_opacity, has_dim);
     // Selection
     foreground = choose_color(float(is_selected & ONE), color_to_vec(highlight_fg), foreground);
     // Underline and strike through (rendered via sprites)
     float in_url = float((is_selected & TWO) >> 1);
     decoration_fg = choose_color(in_url, color_to_vec(url_color), to_color(colors[2], resolved_fg));
-    underline_pos = choose_color(in_url, to_sprite_pos(pos, url_style, ZERO, ZERO), to_sprite_pos(pos, (text_attrs >> 2) & DECORATION_MASK, ZERO, ZERO));
-    strike_pos = to_sprite_pos(pos, ((text_attrs >> 7) & STRIKE_MASK) * FOUR, ZERO, ZERO);
+    underline_pos = choose_color(in_url, to_sprite_pos(pos, url_style, ZERO, ZERO), to_sprite_pos(pos, (text_attrs >> DECORATION_SHIFT) & THREE, ZERO, ZERO));
+    strike_pos = to_sprite_pos(pos, ((text_attrs >> STRIKE_SHIFT) & ONE) * FOUR, ZERO, ZERO);
 
     // Cursor
     foreground = choose_color(cursor, bg, foreground);

kitty/child-monitor.c

@@ -502,7 +502,7 @@ collect_cursor_info(CursorRenderInfo *ans, Window *w, double now, OSWindow *os_w
     ans->is_visible = false;
     if (rd->screen->scrolled_by || !screen_is_cursor_visible(rd->screen)) return;
     double time_since_start_blink = now - os_window->cursor_blink_zero_time;
-    bool cursor_blinking = OPT(cursor_blink_interval) > 0 && os_window->is_focused && time_since_start_blink <= OPT(cursor_stop_blinking_after) ? true : false;
+    bool cursor_blinking = OPT(cursor_blink_interval) > 0 && os_window->is_focused && (OPT(cursor_stop_blinking_after) == 0 || time_since_start_blink <= OPT(cursor_stop_blinking_after));
     bool do_draw_cursor = true;
     if (cursor_blinking) {
         int t = (int)(time_since_start_blink * 1000);
@@ -572,6 +572,10 @@ prepare_to_render_os_window(OSWindow *os_window, double now, unsigned int *activ
     bool needs_render = os_window->needs_render;
     os_window->needs_render = false;
     if (TD.screen && os_window->num_tabs > 1) {
+        if (!os_window->tab_bar_data_updated) {
+            call_boss(update_tab_bar_data, "K", os_window->id);
+            os_window->tab_bar_data_updated = true;
+        }
         if (send_cell_data_to_gpu(TD.vao_idx, 0, TD.xstart, TD.ystart, TD.dx, TD.dy, TD.screen, os_window)) needs_render = true;
     }
     if (OPT(mouse_hide_wait) > 0 && now - os_window->last_mouse_activity_at > OPT(mouse_hide_wait)) hide_mouse(os_window);
@@ -735,6 +739,30 @@ process_pending_resizes(double now) {
     }
 }
 
+static inline void
+close_all_windows() {
+    for (size_t w = 0; w < global_state.num_os_windows; w++) mark_os_window_for_close(&global_state.os_windows[w], true);
+    global_state.close_all_windows = false;
+}
+
+static inline bool
+process_pending_closes(ChildMonitor *self) {
+    bool has_open_windows = false;
+    for (size_t w = global_state.num_os_windows; w > 0; w--) {
+        OSWindow *os_window = global_state.os_windows + w - 1;
+        if (should_os_window_close(os_window)) {
+            destroy_os_window(os_window);
+            call_boss(on_os_window_closed, "Kii", os_window->id, os_window->window_width, os_window->window_height);
+            for (size_t t=0; t < os_window->num_tabs; t++) {
+                Tab *tab = os_window->tabs + t;
+                for (size_t w = 0; w < tab->num_windows; w++) mark_child_for_close(self, tab->windows[w].id);
+            }
+            remove_os_window(os_window->id);
+        } else has_open_windows = true;
+    }
+    return has_open_windows;
+}
+
 static PyObject*
 main_loop(ChildMonitor *self, PyObject *a UNUSED) {
 #define main_loop_doc "The main thread loop"
@@ -746,23 +774,8 @@ main_loop(ChildMonitor *self, PyObject *a UNUSED) {
         render(now);
         wait_for_events();
         parse_input(self);
-        if (global_state.close_all_windows) {
-            for (size_t w = 0; w < global_state.num_os_windows; w++) mark_os_window_for_close(&global_state.os_windows[w], true);
-            global_state.close_all_windows = false;
-        }
-        has_open_windows = false;
-        for (size_t w = global_state.num_os_windows; w > 0; w--) {
-            OSWindow *os_window = global_state.os_windows + w - 1;
-            if (should_os_window_close(os_window)) {
-                destroy_os_window(os_window);
-                call_boss(on_os_window_closed, "Kii", os_window->id, os_window->window_width, os_window->window_height);
-                for (size_t t=0; t < os_window->num_tabs; t++) {
-                    Tab *tab = os_window->tabs + t;
-                    for (size_t w = 0; w < tab->num_windows; w++) mark_child_for_close(self, tab->windows[w].id);
-                }
-                remove_os_window(os_window->id);
-            } else has_open_windows = true;
-        }
+        if (global_state.close_all_windows) close_all_windows();
+        has_open_windows = process_pending_closes(self);
     }
     if (PyErr_Occurred()) return NULL;
     Py_RETURN_NONE;

kitty/cli.py

@@ -4,13 +4,32 @@
 
 import os
 import re
-import subprocess
 import sys
 from collections import deque
 
 from .config import defaults, load_config
+from .config_utils import resolve_config
 from .constants import appname, defconf, is_macos, is_wayland, str_version
-from .layout import all_layouts
+
+CONFIG_HELP = '''\
+Specify a path to the configuration file(s) to use. All configuration files are
+merged onto the builtin {conf_name}.conf, overriding the builtin values. This option
+can be specified multiple times to read multiple configuration files in
+sequence, which are merged. Use the special value NONE to not load a config
+file.
+
+If this option is not specified, config files are searched for in the order:
+"$XDG_CONFIG_HOME/{appname}/{conf_name}.conf", "~/.config/{appname}/{conf_name}.conf", {macos_confpath}
+"$XDG_CONFIG_DIRS/{appname}/{conf_name}.conf". The first one that exists is used as the
+config file.
+
+If the environment variable "KITTY_CONFIG_DIRECTORY" is specified, that
+directory is always used and the above searching does not happen.
+
+If "/etc/xdg/{appname}/{conf_name}.conf" exists it is merged before (i.e. with lower
+priority) than any user config files. It can be used to specify system-wide
+defaults for all users.
+'''.replace('{macos_confpath}', '~/Library/Preferences/{appname}/{conf_name}.conf, ' if is_macos else '')
 
 OPTIONS = '''
 --class
@@ -30,24 +49,9 @@ Set the window title. This will override any title set by the program running in
 only use this if you are running a program that does not set titles.
 
 
---config
+--config -c
 type=list
-Specify a path to the configuration file(s) to use.
-Can be specified multiple times to read multiple configuration files in
-sequence, which are merged.  Use the special value NONE to not load a config
-file.
-
-If this option is not specified, config files are searched for in the order:
-"$XDG_CONFIG_HOME/kitty/kitty.conf", "~/.config/kitty/kitty.conf", {macos_confpath}
-"$XDG_CONFIG_DIRS/kitty/kitty.conf". The first one that exists is used as the
-config file.
-
-If the environment variable "KITTY_CONFIG_DIRECTORY" is specified, that
-directory is always used and the above searching does not happen.
-
-If "/etc/xdg/kitty/kitty.conf" exists it is used as a base config file onto
-which any user config files are merged.
-
+{config_help}
 
 --override -o
 type=list
@@ -55,10 +59,6 @@ Override individual configuration options, can be specified multiple times.
 Syntax: |_ name=value|. For example: |_ -o font_size=20|
 
 
---cmd -c
-Run python code in the kitty context
-
-
 --directory -d
 default=.
 Change to the specified directory when launching
@@ -70,12 +70,6 @@ condition=not is_macos
 Detach from the controlling terminal, if any
 
 
---window-layout
-type=choices
-choices={window_layout_choices}
-The window layout to use on startup.
-
-
 --session
 Path to a file containing the startup |_ session| (tabs, windows, layout, programs).
 See the README file for details and an example.
@@ -118,8 +112,10 @@ Output commands received from child process to stdout
 
 
 --replay-commands
-type=bool-set
-Replay previously dumped commands
+Replay previously dumped commands. Specify the path to a dump file previously created by --dump-commands. You
+can open a new kitty window to replay the commands with:
+
+    kitty sh -c "kitty --replay-commands /path/to/dump/file; read"
 
 
 --dump-bytes
@@ -139,6 +135,11 @@ Print out information about the selection of fallback fonts for characters not p
 --debug-config
 type=bool-set
 Print out information about the system and kitty configuration.
+
+
+--execute -e
+type=bool-set
+!
 '''
 
 
@@ -245,8 +246,12 @@ def prettify(text):
     return text
 
 
-def version():
-    return '{} {} created by {}'.format(italic(appname), green(str_version), title('Kovid Goyal'))
+def version(add_rev=False):
+    rev = ''
+    from . import fast_data_types
+    if add_rev and hasattr(fast_data_types, 'KITTY_VCS_REV'):
+        rev = ' ({})'.format(fast_data_types.KITTY_VCS_REV[:10])
+    return '{} {}{} created by {}'.format(italic(appname), green(str_version), rev, title('Kovid Goyal'))
 
 
 def wrap(text, limit=80):
@@ -283,7 +288,8 @@ def wrap(text, limit=80):
 
 
 def print_help_for_seq(seq, usage, message, appname):
-    from kitty.icat import screen_size
+    from kitty.utils import screen_size_function
+    screen_size = screen_size_function()
     try:
         linesz = min(screen_size().cols, 76)
     except EnvironmentError:
@@ -319,12 +325,15 @@ def print_help_for_seq(seq, usage, message, appname):
         if isinstance(opt, str):
             a('{}:'.format(title(opt)))
             continue
+        help_text = opt['help']
+        if help_text == '!':
+            continue  # hidden option
         a('  ' + ', '.join(map(green, sorted(opt['aliases']))))
         if not opt.get('type', '').startswith('bool-'):
             blocks[-1] += '={}'.format(italic(opt['dest'].upper()))
         if opt.get('help'):
             defval = opt.get('default')
-            t = opt['help'].replace('%default', str(defval))
+            t = help_text.replace('%default', str(defval))
             wa(prettify(t.strip()), indent=4)
             if defval is not None:
                 wa('Default: {}'.format(defval), indent=4)
@@ -333,7 +342,8 @@ def print_help_for_seq(seq, usage, message, appname):
             a('')
 
     text = '\n'.join(blocks) + '\n\n' + version()
-    if sys.stdout.isatty():
+    if print_help_for_seq.allow_pager and sys.stdout.isatty():
+        import subprocess
         p = subprocess.Popen(['less', '-isRXF'], stdin=subprocess.PIPE)
         p.communicate(text.encode('utf-8'))
         raise SystemExit(p.wait())
@@ -341,6 +351,9 @@ def print_help_for_seq(seq, usage, message, appname):
         print(text)
 
 
+print_help_for_seq.allow_pager = True
+
+
 def defval_for_opt(opt):
     dv = opt.get('default')
     typ = opt.get('type', '')
@@ -470,8 +483,8 @@ def parse_cmdline(oc, disabled, args=None):
 def options_spec():
     if not hasattr(options_spec, 'ans'):
         options_spec.ans = OPTIONS.format(
-            appname=appname, macos_confpath='~/Library/Preferences/kitty/kitty.conf' if is_macos else '',
-            window_layout_choices=', '.join(all_layouts)
+            appname=appname, config_help=CONFIG_HELP.format(appname=appname, conf_name=appname)
+
         )
     return options_spec.ans
 
@@ -486,70 +499,82 @@ def parse_args(args=None, ospec=options_spec, usage=None, message=None, appname=
 SYSTEM_CONF = '/etc/xdg/kitty/kitty.conf'
 
 
-def resolve_config(config_files_on_cmd_line):
-    if config_files_on_cmd_line:
-        if 'NONE' not in config_files_on_cmd_line:
-            yield SYSTEM_CONF
-            for cf in config_files_on_cmd_line:
-                yield cf
-    else:
-        yield SYSTEM_CONF
-        yield defconf
-
-
-def print_shortcut(key, action):
+def print_shortcut(key_sequence, action):
     if not getattr(print_shortcut, 'maps', None):
         from kitty.keys import defines
         v = vars(defines)
-        mmap = {m.split('_')[-1].lower(): x for m, x in v.items() if m.startswith('GLFW_MOD_')}
-        kmap = {k.split('_')[-1].lower(): x for k, x in v.items() if k.startswith('GLFW_KEY_')}
+        mmap = {m[len('GLFW_MOD_'):].lower(): x for m, x in v.items() if m.startswith('GLFW_MOD_')}
+        kmap = {k[len('GLFW_KEY_'):].lower(): x for k, x in v.items() if k.startswith('GLFW_KEY_')}
         krmap = {v: k for k, v in kmap.items()}
         print_shortcut.maps = mmap, krmap
     mmap, krmap = print_shortcut.maps
-    names = []
-    mods, key = key
-    for name, val in mmap.items():
-        if mods & val:
-            names.append(name)
-    if key:
-        names.append(krmap[key])
-    print('\t', '+'.join(names), action)
+    keys = []
+    for key in key_sequence:
+        names = []
+        mods, key = key
+        for name, val in mmap.items():
+            if mods & val:
+                names.append(name)
+        if key:
+            names.append(krmap[key])
+        keys.append('+'.join(names))
+
+    print('\t', ' > '.join(keys), action)
+
+
+def print_shortcut_changes(defns, text, changes):
+    if changes:
+        print(title(text))
+
+        for k in sorted(changes):
+            print_shortcut(k, defns[k])
 
 
 def compare_keymaps(final, initial):
     added = set(final) - set(initial)
     removed = set(initial) - set(final)
     changed = {k for k in set(final) & set(initial) if final[k] != initial[k]}
-    if added:
-        print(title('Added shortcuts:'))
-        for k in added:
-            print_shortcut(k, final[k])
-    if removed:
-        print(title('Removed shortcuts:'))
-        for k in removed:
-            print_shortcut(k, initial[k])
-    if changed:
-        print(title('Changed shortcuts:'))
-        for k in changed:
-            print_shortcut(k, final[k])
+    print_shortcut_changes(final, 'Added shortcuts:', added)
+    print_shortcut_changes(initial, 'Removed shortcuts:', removed)
+    print_shortcut_changes(final, 'Changed shortcuts:', changed)
+
+
+def flatten_sequence_map(m):
+    ans = {}
+    for k, rest_map in m.items():
+        for r, action in rest_map.items():
+            ans[(k,) + (r)] = action
+    return ans
 
 
 def compare_opts(opts):
     print('\nConfig options different from defaults:')
-    for f in sorted(defaults._fields):
-        if getattr(opts, f) != getattr(defaults, f):
-            if f == 'keymap':
-                compare_keymaps(opts.keymap, defaults.keymap)
-            else:
-                print(title('{:20s}'.format(f)), getattr(opts, f))
+    default_opts = load_config()
+    changed_opts = [
+        f for f in sorted(defaults._fields)
+        if f not in ('key_definitions', 'keymap', 'sequence_map') and getattr(opts, f) != getattr(defaults, f)
+    ]
+    field_len = max(map(len, changed_opts)) if changed_opts else 20
+    fmt = '{{:{:d}s}}'.format(field_len)
+    for f in changed_opts:
+        print(title(fmt.format(f)), getattr(opts, f))
+
+    final, initial = opts.keymap, default_opts.keymap
+    final = {(k,): v for k, v in final.items()}
+    initial = {(k,): v for k, v in initial.items()}
+    final_s, initial_s = map(flatten_sequence_map, (opts.sequence_map, default_opts.sequence_map))
+    final.update(final_s)
+    initial.update(initial_s)
+    compare_keymaps(final, initial)
 
 
 def create_opts(args, debug_config=False):
-    config = tuple(resolve_config(args.config))
+    config = tuple(resolve_config(SYSTEM_CONF, defconf, args.config))
     if debug_config:
-        print(version())
+        print(version(add_rev=True))
         print(' '.join(os.uname()))
         if is_macos:
+            import subprocess
             print(' '.join(subprocess.check_output(['sw_vers']).decode('utf-8').splitlines()).strip())
         else:
             print('Running under:', green('Wayland' if is_wayland else 'X11'))

kitty/client.py

@@ -106,6 +106,14 @@ def screen_backspace():
     write('\x08')
 
 
+def screen_set_cursor(mode, secondary):
+    write(CSI + '%d q' % secondary)
+
+
+def screen_insert_lines(num):
+    write(CSI + '%dL' % num)
+
+
 def draw(*a):
     write(' '.join(a))
 
@@ -130,7 +138,7 @@ set_dynamic_color = set_color_table_color = write_osc
 
 def replay(raw):
     for line in raw.splitlines():
-        if line.strip():
+        if line.strip() and not line.startswith('#'):
             cmd, rest = line.partition(' ')[::2]
             if cmd in {'draw', 'set_title', 'set_icon', 'set_dynamic_color', 'set_color_table_color'}:
                 globals()[cmd](rest)

kitty/cmds.py

@@ -0,0 +1,601 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
+
+import json
+import os
+import sys
+
+from .cli import parse_args
+from .config import parse_config, parse_send_text_bytes
+from .constants import appname
+from .tabs import SpecialWindow
+from .utils import non_blocking_read
+
+
+class MatchError(ValueError):
+
+    hide_traceback = True
+
+    def __init__(self, expression, target='windows'):
+        ValueError.__init__(self, 'No matching {} for expression: {}'.format(target, expression))
+
+
+def cmd(short_desc, desc=None, options_spec=None, no_response=False, argspec='...', string_return_is_error=False):
+
+    def w(func):
+        func.short_desc = short_desc
+        func.argspec = argspec
+        func.desc = desc or short_desc
+        func.name = func.__name__[4:].replace('_', '-')
+        func.options_spec = options_spec
+        func.is_cmd = True
+        func.impl = lambda: globals()[func.__name__[4:]]
+        func.no_response = no_response
+        func.string_return_is_error = string_return_is_error
+        return func
+    return w
+
+
+MATCH_WINDOW_OPTION = '''\
+--match -m
+The window to match. Match specifications are of the form:
+|_ field:regexp|. Where field can be one of: id, title, pid, cwd, cmdline, num.
+You can use the |_ ls| command to get a list of windows. Note that for
+numeric fields such as id, pid and num the expression is interpreted as a number,
+not a regular expression. The field num refers to the window position in the current tab,
+starting from zero and counting clockwise (this is the same as the order in which the
+windows are reported by the |_ ls| command). The window id of the current window
+is available as the KITTY_WINDOW_ID environment variable.
+'''
+MATCH_TAB_OPTION = '''\
+--match -m
+The tab to match. Match specifications are of the form:
+|_ field:regexp|. Where field can be one of: id, title, pid, cwd, cmdline.
+You can use the |_ ls| command to get a list of tabs. Note that for
+numeric fields such as id and pid the expression is interpreted as a number,
+not a regular expression. When using title or id, first a matching tab is
+looked for and if not found a matching window is looked for, and the tab
+for that window is used.
+'''
+
+
+# ls {{{
+@cmd(
+    'List all tabs/windows',
+    'List all windows. The list is returned as JSON tree. The top-level is a list of'
+    ' operating system {appname} windows. Each OS window has an |_ id| and a list'
+    ' of |_ tabs|. Each tab has its own |_ id|, a |_ title| and a list of |_ windows|.'
+    ' Each window has an |_ id|, |_ title|, |_ current working directory|, |_ process id (PID)| and'
+    ' |_ command-line| of the process running in the window.\n\n'
+    'You can use these criteria to select windows/tabs for the other commands.'.format(appname=appname),
+    argspec=''
+)
+def cmd_ls(global_opts, opts, args):
+    pass
+
+
+def ls(boss, window):
+    data = list(boss.list_os_windows())
+    data = json.dumps(data, indent=2, sort_keys=True)
+    return data
+# }}}
+
+
+# set_font_size {{{
+@cmd(
+    'Set the font size in all windows',
+    'Sets the font size to the specified size, in pts.',
+    argspec='FONT_SIZE'
+)
+def cmd_set_font_size(global_opts, opts, args):
+    try:
+        return {'size': float(args[0])}
+    except IndexError:
+        raise SystemExit('No font size specified')
+
+
+def set_font_size(boss, window, payload):
+    boss.set_font_size(payload['size'])
+# }}}
+
+
+# send_text {{{
+@cmd(
+    'Send arbitrary text to specified windows',
+    'Send arbitrary text to specified windows. The text follows Python'
+    ' escaping rules. So you can use escapes like |_ \\x1b| to send control codes'
+    ' and |_ \\u21fa| to send unicode characters. If you use the |_ --match| option'
+    ' the text will be sent to all matched windows. By default, text is sent to'
+    ' only the currently active window.',
+    options_spec=MATCH_WINDOW_OPTION + '''\n
+--stdin
+type=bool-set
+Read the text to be sent from |_ stdin|. Note that in this case the text is sent as is,
+not interpreted for escapes. If stdin is a terminal, you can press Ctrl-D to end reading.
+
+
+--from-file
+Path to a file whose contents you wish to send. Note that in this case the file contents
+are sent as is, not interpreted for escapes.
+''',
+    no_response=True,
+    argspec='[TEXT TO SEND]'
+)
+def cmd_send_text(global_opts, opts, args):
+    limit = 1024
+    ret = {'match': opts.match, 'is_binary': False}
+
+    def pipe(src=sys.stdin):
+        ret['is_binary'] = True
+        import select
+        with non_blocking_read() as fd:
+            keep_going = True
+            while keep_going:
+                rd = select.select([fd], [], [])
+                if rd:
+                    data = sys.stdin.buffer.read()
+                    if not data:
+                        break
+                    data = data.decode('utf-8')
+                    if '\x04' in data:
+                        data = data[:data.index('\x04')]
+                        keep_going = False
+                    while data:
+                        ret['text'] = data[:limit]
+                        yield ret
+                        data = data[limit:]
+                else:
+                    break
+
+    def chunks(text):
+        ret['is_binary'] = False
+        while text:
+            ret['text'] = text[:limit]
+            yield ret
+            text = text[limit:]
+
+    def file_pipe(path):
+        ret['is_binary'] = True
+        with open(path, encoding='utf-8') as f:
+            while True:
+                data = f.read(limit)
+                if not data:
+                    break
+                ret['text'] = data
+                yield ret
+
+    sources = []
+    if opts.stdin:
+        sources.append(pipe())
+
+    if opts.from_file:
+        sources.append(file_pipe(opts.from_file))
+
+    text = ' '.join(args)
+    sources.append(chunks(text))
+
+    def chain():
+        for src in sources:
+            yield from src
+    return chain()
+
+
+def send_text(boss, window, payload):
+    windows = [boss.active_window]
+    match = payload['match']
+    if match:
+        windows = tuple(boss.match_windows(match))
+    data = payload['text'].encode('utf-8') if payload['is_binary'] else parse_send_text_bytes(payload['text'])
+    for window in windows:
+        if window is not None:
+            window.write_to_child(data)
+# }}}
+
+
+# set_window_title {{{
+@cmd(
+    'Set the window title',
+    'Set the title for the specified window(s). If you use the |_ --match| option'
+    ' the title will be set for all matched windows. By default, only the window'
+    ' in which the command is run is affected. If you do not specify a title, the'
+    ' last title set by the child process running in the window will be used.',
+    options_spec=MATCH_WINDOW_OPTION,
+    argspec='TITLE ...'
+)
+def cmd_set_window_title(global_opts, opts, args):
+    return {'title': ' '.join(args), 'match': opts.match}
+
+
+def set_window_title(boss, window, payload):
+    windows = [window or boss.active_window]
+    match = payload['match']
+    if match:
+        windows = tuple(boss.match_windows(match))
+        if not windows:
+            raise MatchError(match)
+    for window in windows:
+        if window:
+            window.set_title(payload['title'])
+# }}}
+
+
+# set_tab_title {{{
+@cmd(
+    'Set the tab title',
+    'Set the title for the specified tab(s). If you use the |_ --match| option'
+    ' the title will be set for all matched tabs. By default, only the tab'
+    ' in which the command is run is affected. If you do not specify a title, the'
+    ' title of the currently active window in the tab is used.',
+    options_spec=MATCH_TAB_OPTION,
+    argspec='TITLE ...'
+)
+def cmd_set_tab_title(global_opts, opts, args):
+    return {'title': ' '.join(args), 'match': opts.match}
+
+
+def set_tab_title(boss, window, payload):
+    match = payload['match']
+    if match:
+        tabs = tuple(boss.match_tabs(match))
+        if not tabs:
+            raise MatchError(match, 'tabs')
+    else:
+        tabs = [boss.tab_for_window(window) if window else boss.active_tab]
+    for tab in tabs:
+        if tab:
+            tab.set_title(payload['title'])
+# }}}
+
+
+# close_window {{{
+@cmd(
+    'Close the specified window(s)',
+    options_spec=MATCH_WINDOW_OPTION + '''\n
+--self
+type=bool-set
+If specified close the window this command is run in, rather than the active window.
+''',
+    argspec=''
+)
+def cmd_close_window(global_opts, opts, args):
+    return {'match': opts.match, 'self': opts.self}
+
+
+def close_window(boss, window, payload):
+    match = payload['match']
+    if match:
+        windows = tuple(boss.match_windows(match))
+        if not windows:
+            raise MatchError(match)
+    else:
+        windows = [window if window and payload['self'] else boss.active_window]
+    for window in windows:
+        if window:
+            boss.close_window(window)
+# }}}
+
+
+# resize_window {{{
+@cmd(
+    'Resize the specified window',
+    'Resize the specified window in the current layout. Note that not all layouts can resize all windows in all directions.',
+    options_spec=MATCH_WINDOW_OPTION + '''\n
+--increment -i
+type=int
+default=2
+The number of cells to change the size by, can be negative to decrease the size.
+
+
+--axis -a
+type=choices
+choices=horizontal,vertical,reset
+default=horizontal
+The axis along which to resize. If |_ horizontal|, it will make the window wider or narrower by the specified increment.
+If |_ vertical|, it will make the window taller or shorter by the specified increment. The special value |_ reset| will
+reset the layout to its default configuration.
+
+
+--self
+type=bool-set
+If specified close the window this command is run in, rather than the active window.
+''',
+    argspec='',
+    string_return_is_error=True
+)
+def cmd_resize_window(global_opts, opts, args):
+    return {'match': opts.match, 'increment': opts.increment, 'axis': opts.axis, 'self': opts.self}
+
+
+def resize_window(boss, window, payload):
+    match = payload['match']
+    if match:
+        windows = tuple(boss.match_windows(match))
+        if not windows:
+            raise MatchError(match)
+    else:
+        windows = [window if window and payload['self'] else boss.active_window]
+    resized = False
+    if windows and windows[0]:
+        resized = boss.resize_layout_window(
+            windows[0], increment=payload['increment'], is_horizontal=payload['axis'] == 'horizontal',
+            reset=payload['axis'] == 'reset'
+        )
+    return resized
+# }}}
+
+
+# close_tab {{{
+@cmd(
+    'Close the specified tab(s)',
+    options_spec=MATCH_TAB_OPTION + '''\n
+--self
+type=bool-set
+If specified close the tab this command is run in, rather than the active tab.
+''',
+    argspec=''
+)
+def cmd_close_tab(global_opts, opts, args):
+    return {'match': opts.match, 'self': opts.self}
+
+
+def close_tab(boss, window, payload):
+    match = payload['match']
+    if match:
+        tabs = tuple(boss.match_tabs(match))
+        if not tabs:
+            raise MatchError(match, 'tabs')
+    else:
+        tabs = [boss.tab_for_window(window) if window and payload['self'] else boss.active_tab]
+    for tab in tabs:
+        if window:
+            if tab:
+                boss.close_tab(tab)
+# }}}
+
+
+# new_window {{{
+@cmd(
+    'Open new window',
+    'Open a new window in the specified tab. If you use the |_ --match| option'
+    ' the first matching tab is used. Otherwise the currently active tab is used.'
+    ' Prints out the id of the newly opened window. Any command line arguments'
+    ' are assumed to be the command line used to run in the new window, if none'
+    ' are provided, the default shell is run. For example:\n'
+    '|_ kitty @ new-window --title Email mutt|',
+    options_spec=MATCH_TAB_OPTION + '''\n
+--title
+The title for the new window. By default it will use the title set by the
+program running in it.
+
+
+--cwd
+The initial working directory for the new window.
+
+
+--keep-focus
+type=bool-set
+Keep the current window focused instead of switching to the newly opened window
+
+
+--new-tab
+type=bool-set
+Open a new tab
+
+
+--tab-title
+When using --new-tab set the title of the tab.
+''',
+    argspec='[CMD ...]'
+)
+def cmd_new_window(global_opts, opts, args):
+    return {'match': opts.match, 'title': opts.title, 'cwd': opts.cwd,
+            'new_tab': opts.new_tab, 'tab_title': opts.tab_title,
+            'keep_focus': opts.keep_focus, 'args': args or []}
+
+
+def new_window(boss, window, payload):
+    w = SpecialWindow(cmd=payload['args'] or None, override_title=payload['title'], cwd=payload['cwd'])
+    old_window = boss.active_window
+    if payload['new_tab']:
+        boss._new_tab(w)
+        tab = boss.active_tab
+        if payload['tab_title']:
+            tab.set_title(payload['tab_title'])
+        wid = boss.active_window.id
+        if payload['keep_focus'] and old_window:
+            boss.set_active_window(old_window)
+        return str(wid)
+
+    match = payload['match']
+    if match:
+        tabs = tuple(boss.match_tabs(match))
+        if not tabs:
+            raise MatchError(match, 'tabs')
+    else:
+        tabs = [boss.active_tab]
+    tab = tabs[0]
+    w = tab.new_special_window(w)
+    if payload['keep_focus'] and old_window:
+        boss.set_active_window(old_window)
+    return str(w.id)
+# }}}
+
+
+# focus_window {{{
+@cmd(
+    'Focus the specified window',
+    'Focus the specified window, if no window is specified, focus the window this command is run inside.',
+    options_spec=MATCH_WINDOW_OPTION,
+    argspec='',
+)
+def cmd_focus_window(global_opts, opts, args):
+    return {'match': opts.match}
+
+
+def focus_window(boss, window, payload):
+    windows = [window or boss.active_window]
+    match = payload['match']
+    if match:
+        windows = tuple(boss.match_windows(match))
+        if not windows:
+            raise MatchError(match)
+    for window in windows:
+        if window:
+            boss.set_active_window(window)
+            break
+# }}}
+
+
+# focus_tab {{{
+@cmd(
+    'Focus the specified tab',
+    'The active window in the specified tab will be focused.',
+    options_spec=MATCH_TAB_OPTION,
+    argspec='',
+)
+def cmd_focus_tab(global_opts, opts, args):
+    return {'match': opts.match}
+
+
+def focus_tab(boss, window, payload):
+    match = payload['match']
+    tabs = tuple(boss.match_tabs(match))
+    if not tabs:
+        raise MatchError(match, 'tabs')
+    tab = tabs[0]
+    boss.set_active_tab(tab)
+# }}}
+
+
+# get_text {{{
+@cmd(
+    'Get text from the specified window',
+    options_spec=MATCH_WINDOW_OPTION + '''\n
+--extent
+default=screen
+choices=screen, all, selection
+What text to get. The default of screen means all text currently on the screen. all means
+all the screen+scrollback and selection means currently selected text.
+
+
+--ansi
+type=bool-set
+By default, only plain text is returned. If you specify this flag, the text will
+include the formatting escape codes for colors/bold/italic/etc. Note that when
+getting the current selection, the result is always plain text.
+
+
+--self
+type=bool-set
+If specified get text from the window this command is run in, rather than the active window.
+''',
+    argspec=''
+)
+def cmd_get_text(global_opts, opts, args):
+    return {'match': opts.match, 'extent': opts.extent, 'ansi': opts.ansi, 'self': opts.self}
+
+
+def get_text(boss, window, payload):
+    match = payload['match']
+    if match:
+        windows = tuple(boss.match_windows(match))
+        if not windows:
+            raise MatchError(match)
+    else:
+        windows = [window if window and payload['self'] else boss.active_window]
+    window = windows[0]
+    if payload['extent'] == 'selection':
+        ans = window.text_for_selection()
+    else:
+        ans = window.as_text(as_ansi=bool(payload['ansi']), add_history=True)
+    return ans
+# }}}
+
+
+# set_colors {{{
+@cmd(
+    'Set terminal colors',
+    'Set the terminal colors for the specified windows/tabs (defaults to active window). You can either specify the path to a conf file'
+    ' (in the same format as kitty.conf) to read the colors from or you can specify individual colors,'
+    ' for example: kitty @ set-colors foreground=red background=white',
+    options_spec='''\
+--all -a
+type=bool-set
+By default, colors are only changed for the currently active window. This option will
+cause colors to be changed in all windows.
+
+
+--configured -c
+type=bool-set
+Also change the configured colors (i.e. the colors kitty will use for new
+windows or after a reset).
+
+
+--reset
+type=bool-set
+Restore all colors to the values they had at kitty startup. Note that if you specify
+this option, any color arguments are ignored and --configured and --all are implied.
+''' + '\n\n' + MATCH_WINDOW_OPTION + '\n\n' + MATCH_TAB_OPTION.replace('--match -m', '--match-tab -t'),
+    argspec='COLOR_OR_FILE ...'
+)
+def cmd_set_colors(global_opts, opts, args):
+    from .rgb import color_as_int, Color
+    colors = {}
+    if not opts.reset:
+        for spec in args:
+            if '=' in spec:
+                colors.update(parse_config((spec.replace('=', ' '),)))
+            else:
+                with open(os.path.expanduser(spec), encoding='utf-8', errors='replace') as f:
+                    colors.update(parse_config(f))
+        colors = {k: color_as_int(v) for k, v in colors.items() if isinstance(v, Color)}
+    return {
+            'title': ' '.join(args), 'match_window': opts.match, 'match_tab': opts.match_tab,
+            'all': opts.all or opts.reset, 'configured': opts.configured or opts.reset, 'colors': colors, 'reset': opts.reset
+    }
+
+
+def set_colors(boss, window, payload):
+    from .rgb import color_as_int
+    if payload['all']:
+        windows = tuple(boss.all_windows)
+    else:
+        windows = (window or boss.active_window,)
+        if payload['match_window']:
+            windows = tuple(boss.match_windows(payload['match_window']))
+            if not windows:
+                raise MatchError(payload['match_window'])
+        if payload['match_tab']:
+            tabs = tuple(boss.match_tabs(payload['match_tab']))
+            if not tabs:
+                raise MatchError(payload['match_tab'], 'tabs')
+            for tab in tabs:
+                windows += tuple(tab)
+    if payload['reset']:
+        payload['colors'] = {k: color_as_int(v) for k, v in boss.startup_colors.items()}
+    profiles = tuple(w.screen.color_profile for w in windows)
+    from .fast_data_types import patch_color_profiles
+    patch_color_profiles(payload['colors'], profiles, payload['configured'])
+    boss.patch_colors(payload['colors'], payload['configured'])
+    default_bg_changed = 'background' in payload['colors']
+    for w in windows:
+        if default_bg_changed:
+            boss.default_bg_changed_for(w.id)
+        w.refresh()
+# }}}
+
+
+cmap = {v.name: v for v in globals().values() if hasattr(v, 'is_cmd')}
+
+
+def parse_subcommand_cli(func, args):
+    opts, items = parse_args(args[1:], (func.options_spec or '\n').format, func.argspec, func.desc, '{} @ {}'.format(appname, func.name))
+    return opts, items
+
+
+def display_subcommand_help(func):
+    try:
+        parse_args(['--help'], (func.options_spec or '\n').format, func.argspec, func.desc, func.name)
+    except SystemExit:
+        pass

kitty/cocoa_window.m

@@ -23,8 +23,6 @@ typedef void* rusage_info_t;  // needed for libproc.h
 #endif
 
 static NSMenuItem* title_menu = NULL;
-static bool change_titlebar_color = false;
-static color_type titlebar_color = 0;
 
 
 static NSString*
@@ -207,32 +205,14 @@ cwd_of_process(PyObject *self UNUSED, PyObject *pid_) {
     return PyUnicode_FromString(vpi.pvi_cdir.vip_path);
 }
 
-static inline color_type
-color_as_int(PyObject *color) {
-#define I(n, s) ((PyLong_AsUnsignedLong(PyTuple_GET_ITEM(color, n)) & 0xff) << s)
-    return (I(0, 16) | I(1, 8) | I(2, 0)) & 0xffffff;
-#undef I
-}
-
-static PyObject*
-macos_change_titlebar_color(PyObject *self UNUSED, PyObject *val) {
-    if (val == Py_None) change_titlebar_color = false;
-    else if (val == Py_True) {
-        change_titlebar_color = true;
-        titlebar_color = OPT(background);
-    } else {
-        if (!PyTuple_Check(val)) { PyErr_SetString(PyExc_TypeError, "Not a color tuple"); return NULL; }
-        change_titlebar_color = true;
-        titlebar_color = color_as_int(val);
-    }
-    Py_RETURN_NONE;
+void
+cocoa_set_hide_from_tasks(void) {
+    [NSApp setActivationPolicy:NSApplicationActivationPolicyAccessory];
 }
 
 void
-cocoa_set_titlebar_color(void *w)
+cocoa_set_titlebar_color(void *w, color_type titlebar_color)
 {
-    if (!change_titlebar_color) return;
-
     NSWindow *window = (NSWindow *)w;
 
     double red = ((titlebar_color >> 16) & 0xFF) / 255.0;
@@ -259,7 +239,6 @@ cocoa_set_titlebar_color(void *w)
 static PyMethodDef module_methods[] = {
     {"cocoa_get_lang", (PyCFunction)cocoa_get_lang, METH_NOARGS, ""},
     {"cwd_of_process", (PyCFunction)cwd_of_process, METH_O, ""},
-    {"macos_change_titlebar_color", (PyCFunction)macos_change_titlebar_color, METH_O, ""},
     {NULL, NULL, 0, NULL}        /* Sentinel */
 };
 

kitty/colors.c

@@ -5,6 +5,7 @@
  * Distributed under terms of the GPL3 license.
  */
 
+#define EXTRA_INIT if (PyModule_AddFunctions(module, module_methods) != 0) return false;
 #include "data-types.h"
 #include <structmember.h>
 
@@ -29,23 +30,29 @@ static uint32_t FG_BG_256[256] = {
     0xffffff,  // 15
 };
 
-PyObject* create_256_color_table() {
-    // colors 16..232: the 6x6x6 color cube
-    const uint8_t valuerange[6] = {0x00, 0x5f, 0x87, 0xaf, 0xd7, 0xff};
-    uint8_t i, j=16;
-    for(i = 0; i < 217; i++, j++) {
-        uint8_t r = valuerange[(i / 36) % 6], g = valuerange[(i / 6) % 6], b = valuerange[i % 6];
-        FG_BG_256[j] = (r << 16) | (g << 8) | b;
-    }
-    // colors 233..255: grayscale
-    for(i = 1; i < 24; i++, j++) {
-        uint8_t v = 8 + i * 10;
-        FG_BG_256[j] = (v << 16) | (v << 8) | v;
+static inline void
+init_FG_BG_table() {
+    if (UNLIKELY(FG_BG_256[255] == 0)) {
+        // colors 16..232: the 6x6x6 color cube
+        const uint8_t valuerange[6] = {0x00, 0x5f, 0x87, 0xaf, 0xd7, 0xff};
+        uint8_t i, j=16;
+        for(i = 0; i < 217; i++, j++) {
+            uint8_t r = valuerange[(i / 36) % 6], g = valuerange[(i / 6) % 6], b = valuerange[i % 6];
+            FG_BG_256[j] = (r << 16) | (g << 8) | b;
+        }
+        // colors 233..255: grayscale
+        for(i = 1; i < 24; i++, j++) {
+            uint8_t v = 8 + i * 10;
+            FG_BG_256[j] = (v << 16) | (v << 8) | v;
+        }
     }
+}
 
-    PyObject *ans = PyTuple_New(255);
+PyObject* create_256_color_table() {
+    init_FG_BG_table();
+    PyObject *ans = PyTuple_New(arraysz(FG_BG_256));
     if (ans == NULL) return PyErr_NoMemory();
-    for (i=0; i < 255; i++) {
+    for (size_t i=0; i < arraysz(FG_BG_256); i++) {
         PyObject *temp = PyLong_FromUnsignedLong(FG_BG_256[i]);
         if (temp == NULL) { Py_CLEAR(ans); return NULL; }
         PyTuple_SET_ITEM(ans, i, temp);
@@ -59,7 +66,7 @@ new(PyTypeObject *type, PyObject UNUSED *args, PyObject UNUSED *kwds) {
 
     self = (ColorProfile *)type->tp_alloc(type, 0);
     if (self != NULL) {
-        if (FG_BG_256[255] == 0) create_256_color_table();
+        init_FG_BG_table();
         memcpy(self->color_table, FG_BG_256, sizeof(FG_BG_256));
         memcpy(self->orig_color_table, FG_BG_256, sizeof(FG_BG_256));
         self->dirty = true;
@@ -80,12 +87,10 @@ alloc_color_profile() {
 
 static PyObject*
 update_ansi_color_table(ColorProfile *self, PyObject *val) {
-#define update_ansi_color_table_doc "Update the 16 basic colors"
-    index_type i;
-
+#define update_ansi_color_table_doc "Update the 256 basic colors"
     if (!PyList_Check(val)) { PyErr_SetString(PyExc_TypeError, "color table must be a list"); return NULL; }
-    if (PyList_GET_SIZE(val) != 16) { PyErr_SetString(PyExc_TypeError, "color table must have 16 items"); return NULL; }
-    for (i = 0; i < 16; i++) {
+    if (PyList_GET_SIZE(val) != arraysz(FG_BG_256)) { PyErr_SetString(PyExc_TypeError, "color table must have 256 items"); return NULL; }
+    for (size_t i = 0; i < arraysz(FG_BG_256); i++) {
         self->color_table[i] = PyLong_AsUnsignedLong(PyList_GET_ITEM(val, i));
         self->orig_color_table[i] = self->color_table[i];
     }
@@ -93,6 +98,43 @@ update_ansi_color_table(ColorProfile *self, PyObject *val) {
     Py_RETURN_NONE;
 }
 
+static PyObject*
+patch_color_profiles(PyObject *module UNUSED, PyObject *args) {
+    PyObject *spec, *profiles, *v; ColorProfile *self; int change_configured;
+    if (!PyArg_ParseTuple(args, "O!O!p", &PyDict_Type, &spec, &PyTuple_Type, &profiles, &change_configured)) return NULL;
+    char key[32] = {0};
+    for (size_t i = 0; i < arraysz(FG_BG_256); i++) {
+        snprintf(key, sizeof(key) - 1, "color%zu", i);
+        v = PyDict_GetItemString(spec, key);
+        if (v) {
+            color_type color = PyLong_AsUnsignedLong(v);
+            for (Py_ssize_t i = 0; i < PyTuple_GET_SIZE(profiles); i++) {
+                self = (ColorProfile*)PyTuple_GET_ITEM(profiles, i);
+                self->color_table[i] = color;
+                if (change_configured) self->orig_color_table[i] = color;
+                self->dirty = true;
+            }
+        }
+    }
+#define S(config_name, profile_name) { \
+    v = PyDict_GetItemString(spec, #config_name); \
+    if (v) { \
+        color_type color = PyLong_AsUnsignedLong(v); \
+        for (Py_ssize_t i = 0; i < PyTuple_GET_SIZE(profiles); i++) { \
+            self = (ColorProfile*)PyTuple_GET_ITEM(profiles, i); \
+            self->overridden.profile_name = (color << 8) | 2; \
+            if (change_configured) self->configured.profile_name = color; \
+            self->dirty = true; \
+        } \
+    } \
+}
+        S(foreground, default_fg); S(background, default_bg); S(cursor, cursor_color);
+        S(selection_foreground, highlight_fg); S(selection_background, highlight_bg);
+#undef S
+
+    Py_RETURN_NONE;
+}
+
 color_type
 colorprofile_to_color(ColorProfile *self, color_type entry, color_type defval) {
     color_type t = entry & 0xFF, r;
@@ -184,6 +226,10 @@ color_table_address(ColorProfile *self, PyObject *a UNUSED) {
     return PyLong_FromVoidPtr((void*)self->color_table);
 }
 
+static PyObject*
+default_color_table(PyObject *self UNUSED, PyObject *args UNUSED) {
+    return create_256_color_table();
+}
 
 // Boilerplate {{{
 
@@ -236,5 +282,12 @@ PyTypeObject ColorProfile_Type = {
     .tp_new = new,
 };
 
+static PyMethodDef module_methods[] = {
+    METHODB(default_color_table, METH_NOARGS),
+    METHODB(patch_color_profiles, METH_VARARGS),
+    {NULL, NULL, 0, NULL}        /* Sentinel */
+};
+
+
 INIT_TYPE(ColorProfile)
 // }}}

kitty/config.py

@@ -2,24 +2,23 @@
 # vim:fileencoding=utf-8
 # License: GPL v3 Copyright: 2016, Kovid Goyal <kovid at kovidgoyal.net>
 
-import ast
 import json
 import os
 import re
-import shlex
 import sys
-import tempfile
 from collections import namedtuple
 from contextlib import contextmanager
 
 from . import fast_data_types as defines
 from .config_utils import (
-    init_config, parse_config_base, positive_float, positive_int, to_bool,
-    to_color, unit_float
+    init_config, load_config as _load_config, merge_dicts, parse_config_base,
+    positive_float, positive_int, python_string, to_bool, to_cmdline, to_color,
+    unit_float
 )
 from .constants import cache_dir, defconf
 from .fast_data_types import CURSOR_BEAM, CURSOR_BLOCK, CURSOR_UNDERLINE
 from .layout import all_layouts
+from .rgb import color_as_int, color_from_int
 from .utils import log_error
 
 MINIMUM_FONT_SIZE = 4
@@ -47,18 +46,20 @@ def to_cursor_shape(x):
         )
 
 
-def parse_mods(parts):
+mod_map = {'CTRL': 'CONTROL', 'CMD': 'SUPER', '⌘': 'SUPER', '⌥': 'ALT', 'OPTION': 'ALT', 'KITTY_MOD': 'KITTY'}
+
+
+def parse_mods(parts, sc):
 
     def map_mod(m):
-        return {'CTRL': 'CONTROL', 'CMD': 'SUPER', '⌘': 'SUPER', '⌥': 'ALT', 'OPTION': 'ALT'}.get(m, m)
+        return mod_map.get(m, m)
 
     mods = 0
     for m in parts:
         try:
             mods |= getattr(defines, 'GLFW_MOD_' + map_mod(m.upper()))
         except AttributeError:
-            log_error('Shortcut: {} has unknown modifier, ignoring'.format(
-                parts.join('+')))
+            log_error('Shortcut: {} has unknown modifier, ignoring'.format(sc))
             return
 
     return mods
@@ -80,12 +81,14 @@ named_keys = {
 
 def parse_shortcut(sc):
     parts = sc.split('+')
-    mods = parse_mods(parts[:-1])
+    mods = parse_mods(parts[:-1], sc)
+    if mods is None:
+        return None, None
     key = parts[-1].upper()
     key = getattr(defines, 'GLFW_KEY_' + named_keys.get(key, key), None)
     if key is not None:
         return mods, key
-    return None, None
+    return mods, None
 
 
 KeyAction = namedtuple('KeyAction', 'func args')
@@ -107,29 +110,72 @@ def parse_key_action(action):
         args = tuple(map(parse_key_action, filter(None, parts)))
     elif func == 'send_text':
         args = rest.split(' ', 1)
-    elif func == 'run_simple_kitten':
-        args = rest.split(' ', 2)
+        if len(args) > 0:
+            try:
+                args[1] = parse_send_text_bytes(args[1])
+            except Exception:
+                log_error('Ignoring invalid send_text string: ' + args[1])
+                args[1] = ''
+    elif func in ('run_kitten', 'run_simple_kitten', 'kitten'):
+        if func == 'kitten':
+            args = rest.split(' ', 1)
+        else:
+            args = rest.split(' ', 2)[1:]
+            func = 'kitten'
     elif func == 'goto_tab':
         args = (max(0, int(rest)), )
-    elif func == 'goto_layout':
+    elif func == 'goto_layout' or func == 'kitty_shell':
         args = [rest]
     elif func == 'set_font_size':
         args = (float(rest),)
     elif func in shlex_actions:
-        args = shlex.split(rest)
+        args = to_cmdline(rest)
     return KeyAction(func, args)
 
 
-def parse_key(val, keymap):
+all_key_actions = set()
+sequence_sep = '>'
+
+
+class KeyDefinition:
+
+    def __init__(self, is_sequence, action, mods, key, rest=()):
+        self.is_sequence = is_sequence
+        self.action = action
+        self.trigger = mods, key
+        self.rest = rest
+
+    def resolve(self, kitty_mod):
+        self.trigger = defines.resolve_key_mods(kitty_mod, self.trigger[0]), self.trigger[1]
+        self.rest = tuple((defines.resolve_key_mods(kitty_mod, mods), key) for mods, key in self.rest)
+
+
+def parse_key(val, key_definitions):
     sc, action = val.partition(' ')[::2]
-    sc, action = sc.strip(), action.strip()
+    sc, action = sc.strip().strip(sequence_sep), action.strip()
     if not sc or not action:
         return
-    mods, key = parse_shortcut(sc)
-    if key is None:
-        log_error('Shortcut: {} has unknown key, ignoring'.format(
-            val))
-        return
+    is_sequence = sequence_sep in sc
+    if is_sequence:
+        trigger = None
+        rest = []
+        for part in sc.split(sequence_sep):
+            mods, key = parse_shortcut(part)
+            if key is None:
+                if mods is not None:
+                    log_error('Shortcut: {} has unknown key, ignoring'.format(sc))
+                return
+            if trigger is None:
+                trigger = mods, key
+            else:
+                rest.append((mods, key))
+        rest = tuple(rest)
+    else:
+        mods, key = parse_shortcut(sc)
+        if key is None:
+            if mods is not None:
+                log_error('Shortcut: {} has unknown key, ignoring'.format(sc))
+            return
     try:
         paction = parse_key_action(action)
     except Exception:
@@ -137,7 +183,11 @@ def parse_key(val, keymap):
             action))
     else:
         if paction is not None:
-            keymap[(mods, key)] = paction
+            all_key_actions.add(paction.func)
+            if is_sequence:
+                key_definitions.append(KeyDefinition(True, paction, trigger[0], trigger[1], rest))
+            else:
+                key_definitions.append(KeyDefinition(False, paction, mods, key))
 
 
 def parse_symbol_map(val):
@@ -173,11 +223,10 @@ def parse_symbol_map(val):
 
 
 def parse_send_text_bytes(text):
-    return ast.literal_eval("'''" + text.replace("'''", "'\\''") + "'''"
-                            ).encode('utf-8')
+    return python_string(text).encode('utf-8')
 
 
-def parse_send_text(val, keymap):
+def parse_send_text(val, key_definitions):
     parts = val.split(' ')
 
     def abort(msg):
@@ -190,21 +239,31 @@ def parse_send_text(val, keymap):
     mode, sc = parts[:2]
     text = ' '.join(parts[2:])
     key_str = '{} send_text {} {}'.format(sc, mode, text)
-    return parse_key(key_str, keymap)
+    return parse_key(key_str, key_definitions)
 
 
 def to_modifiers(val):
-    return parse_mods(val.split('+'))
+    return parse_mods(val.split('+'), val) or 0
+
+
+def uniq(vals, result_type=list):
+    seen = set()
+    seen_add = seen.add
+    return result_type(x for x in vals if x not in seen and not seen_add(x))
 
 
 def to_layout_names(raw):
     parts = [x.strip().lower() for x in raw.split(',')]
-    if '*' in parts:
-        return sorted(all_layouts)
+    ans = []
     for p in parts:
-        if p not in all_layouts:
+        if p == '*':
+            ans.extend(sorted(all_layouts))
+            continue
+        name = p.partition(':')[0]
+        if name not in all_layouts:
             raise ValueError('The window layout {} is unknown'.format(p))
-    return parts
+        ans.append(p)
+    return uniq(ans)
 
 
 def adjust_line_height(x):
@@ -216,10 +275,10 @@ def adjust_line_height(x):
 def macos_titlebar_color(x):
     x = x.strip('"')
     if x == 'system':
-        return
+        return 0
     if x == 'background':
-        return True
-    return to_color(x)
+        return 1
+    return (color_as_int(to_color(x)) << 8) | 2
 
 
 def box_drawing_scale(x):
@@ -264,7 +323,8 @@ type_map = {
     'adjust_line_height': adjust_line_height,
     'adjust_column_width': adjust_line_height,
     'scrollback_lines': positive_int,
-    'scrollback_pager': shlex.split,
+    'scrollback_pager': to_cmdline,
+    'open_url_with': to_cmdline,
     'font_size': to_font_size,
     'font_size_delta': positive_float,
     'focus_follows_mouse': to_bool,
@@ -290,26 +350,35 @@ type_map = {
     'initial_window_width': positive_int,
     'initial_window_height': positive_int,
     'macos_hide_titlebar': to_bool,
+    'macos_hide_from_tasks': to_bool,
     'macos_option_as_alt': to_bool,
     'macos_titlebar_color': macos_titlebar_color,
     'box_drawing_scale': box_drawing_scale,
     'background_opacity': unit_float,
+    'dim_opacity': unit_float,
     'tab_separator': tab_separator,
     'active_tab_font_style': tab_font_style,
     'inactive_tab_font_style': tab_font_style,
     'inactive_text_alpha': unit_float,
     'url_style': url_style,
     'copy_on_select': to_bool,
+    'window_alert_on_bell': to_bool,
     'tab_bar_edge': tab_bar_edge,
+    'bell_on_tab': to_bool,
+    'kitty_mod': to_modifiers,
+    'clear_all_shortcuts': to_bool,
+    'clipboard_control': lambda x: frozenset(x.lower().split()),
+    'window_resize_step_cells': int,
+    'window_resize_step_lines': int,
 }
 
 for name in (
     'foreground background cursor active_border_color inactive_border_color'
-    ' selection_foreground selection_background url_color'
+    ' selection_foreground selection_background url_color bell_border_color'
 ).split():
     type_map[name] = to_color
-for i in range(16):
-    type_map['color%d' % i] = to_color
+for i in range(256):
+    type_map['color{}'.format(i)] = to_color
 for a in ('active', 'inactive'):
     for b in ('foreground', 'background'):
         type_map['%s_tab_%s' % (a, b)] = to_color
@@ -317,15 +386,19 @@ for a in ('active', 'inactive'):
 
 def special_handling(key, val, ans):
     if key == 'map':
-        parse_key(val, ans['keymap'])
+        parse_key(val, ans['key_definitions'])
         return True
     if key == 'symbol_map':
         ans['symbol_map'].update(parse_symbol_map(val))
         return True
     if key == 'send_text':
         # For legacy compatibility
-        parse_send_text(val, ans['keymap'])
+        parse_send_text(val, ans['key_definitions'])
         return True
+    if key == 'clear_all_shortcuts':
+        if to_bool(val):
+            ans['key_definitions'] = [None]
+        return
 
 
 defaults = None
@@ -335,10 +408,7 @@ default_config_path = os.path.join(
 
 
 def parse_config(lines, check_keys=True):
-    ans = {
-        'keymap': {},
-        'symbol_map': {},
-    }
+    ans = {'symbol_map': {}, 'keymap': {}, 'sequence_map': {}, 'key_definitions': []}
     parse_config_base(
         lines,
         defaults,
@@ -350,65 +420,38 @@ def parse_config(lines, check_keys=True):
     return ans
 
 
-Options, defaults = init_config(default_config_path, parse_config)
-actions = frozenset(a.func for a in defaults.keymap.values()) | frozenset(
-    'combine send_text goto_tab goto_layout set_font_size new_tab_with_cwd new_window_with_cwd new_os_window_with_cwd'.
+def parse_defaults(lines, check_keys=False):
+    ans = parse_config(lines, check_keys)
+    dfctl = defines.default_color_table()
+
+    for i in range(16, 256):
+        k = 'color{}'.format(i)
+        ans.setdefault(k, color_from_int(dfctl[i]))
+    return ans
+
+
+Options, defaults = init_config(default_config_path, parse_defaults)
+actions = frozenset(all_key_actions) | frozenset(
+    'run_simple_kitten combine send_text goto_tab goto_layout set_font_size new_tab_with_cwd new_window_with_cwd new_os_window_with_cwd'.
     split()
 )
 no_op_actions = frozenset({'noop', 'no-op', 'no_op'})
 
 
-def merge_keymaps(defaults, newvals):
-    ans = defaults.copy()
-    for k, v in newvals.items():
-        f = v.func
-        if f in no_op_actions:
-            ans.pop(k, None)
-            continue
-        if f in actions:
-            ans[k] = v
-    return ans
-
-
-def merge_dicts(defaults, newvals):
-    ans = defaults.copy()
-    ans.update(newvals)
-    return ans
-
-
 def merge_configs(defaults, vals):
     ans = {}
     for k, v in defaults.items():
         if isinstance(v, dict):
             newvals = vals.get(k, {})
-            if k == 'keymap':
-                ans['keymap'] = merge_keymaps(v, newvals)
-            else:
-                ans[k] = merge_dicts(v, newvals)
+            ans[k] = merge_dicts(v, newvals)
+        elif k == 'key_definitions':
+            ans['key_definitions'] = v + vals.get('key_definitions', [])
         else:
             ans[k] = vals.get(k, v)
     return ans
 
 
-def load_config(*paths, overrides=None) -> Options:
-    ans = defaults._asdict()
-    for path in paths:
-        if not path:
-            continue
-        try:
-            f = open(path, encoding='utf-8', errors='replace')
-        except FileNotFoundError:
-            continue
-        with f:
-            vals = parse_config(f)
-            ans = merge_configs(ans, vals)
-    if overrides is not None:
-        vals = parse_config(overrides)
-        ans = merge_configs(ans, vals)
-    return Options(**ans)
-
-
-def build_ansi_color_table(opts: Options = defaults):
+def build_ansi_color_table(opts=defaults):
 
     def as_int(x):
         return (x[0] << 16) | (x[1] << 8) | x[2]
@@ -416,10 +459,11 @@ def build_ansi_color_table(opts: Options = defaults):
     def col(i):
         return as_int(getattr(opts, 'color{}'.format(i)))
 
-    return list(map(col, range(16)))
+    return list(map(col, range(256)))
 
 
 def atomic_save(data, path):
+    import tempfile
     fd, p = tempfile.mkstemp(dir=os.path.dirname(path), suffix='.tmp')
     try:
         with os.fdopen(fd, 'wb') as f:
@@ -491,3 +535,45 @@ def prepare_config_file_for_editing():
         with open(defconf, 'w') as f:
             f.write(commented_out_default_config())
     return defconf
+
+
+def finalize_keys(opts):
+    defns = []
+    for d in opts.key_definitions:
+        if d is None:  # clear_all_shortcuts
+            defns = []
+        else:
+            defns.append(d)
+    for d in defns:
+        d.resolve(opts.kitty_mod)
+    keymap = {}
+    sequence_map = {}
+
+    for defn in defns:
+        is_no_op = defn.action.func in no_op_actions
+        if defn.is_sequence:
+            keymap.pop(defn.trigger, None)
+            s = sequence_map.setdefault(defn.trigger, {})
+            if is_no_op:
+                s.pop(defn.rest, None)
+                if not s:
+                    del sequence_map[defn.trigger]
+            else:
+                s[defn.rest] = defn.action
+        else:
+            sequence_map.pop(defn.trigger, None)
+            if is_no_op:
+                keymap.pop(defn.trigger, None)
+            else:
+                keymap[defn.trigger] = defn.action
+    opts.keymap = keymap
+    opts.sequence_map = sequence_map
+
+
+def load_config(*paths, overrides=None):
+    opts = _load_config(Options, defaults, parse_config, merge_configs, *paths, overrides=overrides)
+    finalize_keys(opts)
+    if opts.background_opacity < 1.0 and opts.macos_titlebar_color:
+        log_error('Cannot use both macos_titlebar_color and background_opacity')
+        opts.macos_titlebar_color = 0
+    return opts

kitty/config_utils.py

@@ -2,11 +2,13 @@
 # vim:fileencoding=utf-8
 # License: GPL v3 Copyright: 2018, Kovid Goyal <kovid at kovidgoyal.net>
 
+import ast
+import os
 import re
-from collections import namedtuple
+import shlex
 
-from .utils import log_error
 from .rgb import to_color as as_color
+from .utils import log_error
 
 key_pat = re.compile(r'([a-zA-Z][a-zA-Z0-9_-]*)\s+(.+)$')
 
@@ -31,33 +33,140 @@ def to_bool(x):
     return x.lower() in 'y yes true'.split()
 
 
+def to_cmdline(x):
+    ans = shlex.split(x)
+    ans[0] = os.path.expandvars(os.path.expanduser(ans[0]))
+    return ans
+
+
+def python_string(text):
+    return ast.literal_eval("'''" + text.replace("'''", "'\\''") + "'''")
+
+
+def parse_line(line, type_map, special_handling, ans, all_keys, base_path_for_includes):
+    line = line.strip()
+    if not line or line.startswith('#'):
+        return
+    m = key_pat.match(line)
+    if m is not None:
+        key, val = m.groups()
+        if special_handling(key, val, ans):
+            return
+        if key == 'include':
+            val = os.path.expandvars(os.path.expanduser(val.strip()))
+            if not os.path.isabs(val):
+                val = os.path.join(base_path_for_includes, val)
+            try:
+                with open(val, encoding='utf-8', errors='replace') as include:
+                    _parse(include, type_map, special_handling, ans, all_keys)
+            except FileNotFoundError:
+                log_error('Could not find included config file: {}, ignoring'.format(val))
+            except EnvironmentError:
+                log_error('Could not read from included config file: {}, ignoring'.format(val))
+            return
+        if all_keys is not None and key not in all_keys:
+            log_error('Ignoring unknown config key: {}'.format(key))
+            return
+        tm = type_map.get(key)
+        if tm is not None:
+            val = tm(val)
+        ans[key] = val
+
+
+def _parse(lines, type_map, special_handling, ans, all_keys):
+    name = getattr(lines, 'name', None)
+    if name:
+        base_path_for_includes = os.path.dirname(os.path.abspath(name))
+    else:
+        from .constants import config_dir
+        base_path_for_includes = config_dir
+    for line in lines:
+        parse_line(line, type_map, special_handling, ans, all_keys, base_path_for_includes)
+
+
 def parse_config_base(
     lines, defaults, type_map, special_handling, ans, check_keys=True
 ):
-    if check_keys:
-        all_keys = defaults._asdict()
-    for line in lines:
-        line = line.strip()
-        if not line or line.startswith('#'):
+    all_keys = defaults._asdict() if check_keys else None
+    _parse(lines, type_map, special_handling, ans, all_keys)
+
+
+def create_options_class(keys):
+    keys = tuple(sorted(keys))
+    slots = keys + ('_fields',)
+
+    def __init__(self, kw):
+        for k, v in kw.items():
+            setattr(self, k, v)
+
+    def __iter__(self):
+        return iter(keys)
+
+    def __len__(self):
+        return len(keys)
+
+    def __getitem__(self, i):
+        if isinstance(i, int):
+            i = keys[i]
+        try:
+            return getattr(self, i)
+        except AttributeError:
+            raise KeyError('No option named: {}'.format(i))
+
+    def _asdict(self):
+        return {k: getattr(self, k) for k in self._fields}
+
+    def _replace(self, **kw):
+        ans = self._asdict()
+        ans.update(kw)
+        return self.__class__(ans)
+
+    ans = type('Options', (), {
+        '__slots__': slots, '__init__': __init__, '_asdict': _asdict, '_replace': _replace, '__iter__': __iter__,
+        '__len__': __len__, '__getitem__': __getitem__
+    })
+    ans._fields = keys
+    return ans
+
+
+def merge_dicts(defaults, newvals):
+    ans = defaults.copy()
+    ans.update(newvals)
+    return ans
+
+
+def resolve_config(SYSTEM_CONF, defconf, config_files_on_cmd_line):
+    if config_files_on_cmd_line:
+        if 'NONE' not in config_files_on_cmd_line:
+            yield SYSTEM_CONF
+            for cf in config_files_on_cmd_line:
+                yield cf
+    else:
+        yield SYSTEM_CONF
+        yield defconf
+
+
+def load_config(Options, defaults, parse_config, merge_configs, *paths, overrides=None):
+    ans = defaults._asdict()
+    for path in paths:
+        if not path:
             continue
-        m = key_pat.match(line)
-        if m is not None:
-            key, val = m.groups()
-            if special_handling(key, val, ans):
-                continue
-            if check_keys:
-                if key not in all_keys:
-                    log_error('Ignoring unknown config key: {}'.format(key))
-                    continue
-            tm = type_map.get(key)
-            if tm is not None:
-                val = tm(val)
-            ans[key] = val
+        try:
+            f = open(path, encoding='utf-8', errors='replace')
+        except FileNotFoundError:
+            continue
+        with f:
+            vals = parse_config(f)
+            ans = merge_configs(ans, vals)
+    if overrides is not None:
+        vals = parse_config(overrides)
+        ans = merge_configs(ans, vals)
+    return Options(ans)
 
 
 def init_config(defaults_path, parse_config):
     with open(defaults_path, encoding='utf-8', errors='replace') as f:
-        defaults = parse_config(f.read().splitlines(), check_keys=False)
-    Options = namedtuple('Defaults', ','.join(defaults.keys()))
-    defaults = Options(**defaults)
+        defaults = parse_config(f, check_keys=False)
+    Options = create_options_class(defaults.keys())
+    defaults = Options(defaults)
     return Options, defaults

kitty/constants.py

@@ -11,7 +11,7 @@ from collections import namedtuple
 from .fast_data_types import set_boss as set_c_boss
 
 appname = 'kitty'
-version = (0, 8, 4)
+version = (0, 10, 0)
 str_version = '.'.join(map(str, version))
 _plat = sys.platform.lower()
 is_macos = 'darwin' in _plat
@@ -105,3 +105,6 @@ def glfw_path(module):
 is_wayland = False
 if os.environ.get('WAYLAND_DISPLAY') and 'KITTY_ENABLE_WAYLAND' in os.environ and os.path.exists(glfw_path('wayland')):
     is_wayland = True
+
+
+supports_primary_selection = not is_macos and not is_wayland

kitty/cursor.c

@@ -24,7 +24,7 @@ dealloc(Cursor* self) {
 
 #define EQ(x) (a->x == b->x)
 static int __eq__(Cursor *a, Cursor *b) {
-    return EQ(bold) && EQ(italic) && EQ(strikethrough) && EQ(reverse) && EQ(decoration) && EQ(fg) && EQ(bg) && EQ(decoration_fg) && EQ(x) && EQ(y) && EQ(shape) && EQ(blink);
+    return EQ(bold) && EQ(italic) && EQ(strikethrough) && EQ(dim) && EQ(reverse) && EQ(decoration) && EQ(fg) && EQ(bg) && EQ(decoration_fg) && EQ(x) && EQ(y) && EQ(shape) && EQ(blink);
 }
 
 static const char* cursor_names[NUM_OF_CURSOR_SHAPES] = { "NO_SHAPE", "BLOCK", "BEAM", "UNDERLINE" };
@@ -33,16 +33,16 @@ static const char* cursor_names[NUM_OF_CURSOR_SHAPES] = { "NO_SHAPE", "BLOCK", "
 static PyObject *
 repr(Cursor *self) {
     return PyUnicode_FromFormat(
-        "Cursor(x=%u, y=%u, shape=%s, blink=%R, fg=#%08x, bg=#%08x, bold=%R, italic=%R, reverse=%R, strikethrough=%R, decoration=%d, decoration_fg=#%08x)",
+        "Cursor(x=%u, y=%u, shape=%s, blink=%R, fg=#%08x, bg=#%08x, bold=%R, italic=%R, reverse=%R, strikethrough=%R, dim=%R, decoration=%d, decoration_fg=#%08x)",
         self->x, self->y, (self->shape < NUM_OF_CURSOR_SHAPES ? cursor_names[self->shape] : "INVALID"),
-        BOOL(self->blink), self->fg, self->bg, BOOL(self->bold), BOOL(self->italic), BOOL(self->reverse), BOOL(self->strikethrough), self->decoration, self->decoration_fg
+        BOOL(self->blink), self->fg, self->bg, BOOL(self->bold), BOOL(self->italic), BOOL(self->reverse), BOOL(self->strikethrough), BOOL(self->dim), self->decoration, self->decoration_fg
     );
 }
 
 void
 cursor_reset_display_attrs(Cursor *self) {
     self->bg = 0; self->fg = 0; self->decoration_fg = 0;
-    self->decoration = 0; self->bold = false; self->italic = false; self->reverse = false; self->strikethrough = false;
+    self->decoration = 0; self->bold = false; self->italic = false; self->reverse = false; self->strikethrough = false; self->dim = false;
 }
 
 
@@ -85,6 +85,8 @@ START_ALLOW_CASE_RANGE
                 cursor_reset_display_attrs(self);  break;
             case 1:
                 self->bold = true;  break;
+            case 2:
+                self->dim = true; break;
             case 3:
                 self->italic = true;  break;
             case 4:
@@ -98,7 +100,7 @@ START_ALLOW_CASE_RANGE
             case 21:
                 self->decoration = 2; break;
             case 22:
-                self->bold = false;  break;
+                self->bold = false;  self->dim = false; break;
             case 23:
                 self->italic = false;  break;
             case 24:
@@ -138,6 +140,7 @@ apply_sgr_to_cells(Cell *first_cell, unsigned int cell_count, unsigned int *para
 #define RANGE for(unsigned c = 0; c < cell_count; c++, cell++)
 #define SET(shift) RANGE { cell->attrs |= (1 << shift); } break;
 #define RESET(shift) RANGE { cell->attrs &= ~(1 << shift); } break;
+#define RESET2(shift1, shift2) RANGE { cell->attrs &= ~((1 << shift1) | (1 << shift2)); } break;
 #define SETM(val, mask, shift) { RANGE { cell->attrs &= ~(mask << shift); cell->attrs |= ((val) << shift); } break; }
 #define SET_COLOR(which) { color_type color = 0; parse_color(params, &i, count, &color); if (color) { RANGE { cell->which = color; }} } break;
 #define SIMPLE(which, val) RANGE { cell->which = (val); } break;
@@ -153,6 +156,8 @@ apply_sgr_to_cells(Cell *first_cell, unsigned int cell_count, unsigned int *para
                 break;
             case 1:
                 SET(BOLD_SHIFT);
+            case 2:
+                SET(DIM_SHIFT);
             case 3:
                 SET(ITALIC_SHIFT);
             case 4:
@@ -165,7 +170,7 @@ apply_sgr_to_cells(Cell *first_cell, unsigned int cell_count, unsigned int *para
             case 21:
                 SETM(2, DECORATION_MASK, DECORATION_SHIFT);
             case 22:
-                RESET(BOLD_SHIFT);
+                RESET2(DIM_SHIFT, BOLD_SHIFT);
             case 23:
                 RESET(ITALIC_SHIFT);
             case 24:
@@ -199,6 +204,7 @@ END_ALLOW_CASE_RANGE
         }
     }
 #undef RESET
+#undef RESET2
 #undef SET_COLOR
 #undef SET
 #undef SETM
@@ -237,7 +243,11 @@ cursor_as_sgr(Cursor *self, Cursor *prev) {
 #define SZ sizeof(buf) - (p - buf) - 2
 #define P(fmt, ...) { p += snprintf(p, SZ, fmt ";", __VA_ARGS__); }
     char *p = buf;
-    if (self->bold != prev->bold) P("%d", self->bold ? 1 : 22);
+    bool intensity_differs = self->bold != prev->bold || self->dim != prev->dim;
+    if (intensity_differs) {
+        if (!self->bold || !self->dim) { P("%d", 22); }
+        else P("%d;%d", 1, 2);
+    }
     if (self->italic != prev->italic) P("%d", self->italic ? 3 : 23);
     if (self->reverse != prev->reverse) P("%d", self->reverse ? 7 : 27);
     if (self->strikethrough != prev->strikethrough) P("%d", self->strikethrough ? 9 : 29);
@@ -268,7 +278,7 @@ void cursor_reset(Cursor *self) {
 void cursor_copy_to(Cursor *src, Cursor *dest) {
 #define CCY(x) dest->x = src->x;
     CCY(x); CCY(y); CCY(shape); CCY(blink);
-    CCY(bold); CCY(italic); CCY(strikethrough); CCY(reverse); CCY(decoration); CCY(fg); CCY(bg); CCY(decoration_fg);
+    CCY(bold); CCY(italic); CCY(strikethrough); CCY(dim); CCY(reverse); CCY(decoration); CCY(fg); CCY(bg); CCY(decoration_fg);
 }
 
 static PyObject*
@@ -281,6 +291,7 @@ BOOL_GETSET(Cursor, bold)
 BOOL_GETSET(Cursor, italic)
 BOOL_GETSET(Cursor, reverse)
 BOOL_GETSET(Cursor, strikethrough)
+BOOL_GETSET(Cursor, dim)
 BOOL_GETSET(Cursor, blink)
 
 static PyMemberDef members[] = {
@@ -299,6 +310,7 @@ static PyGetSetDef getseters[] = {
     GETSET(italic)
     GETSET(reverse)
     GETSET(strikethrough)
+    GETSET(dim)
     GETSET(blink)
     {NULL}  /* Sentinel */
 };

kitty/data-types.c

@@ -193,16 +193,16 @@ PyInit_fast_data_types(void) {
 #endif
         if (!init_fonts(m)) return NULL;
 
-#define OOF(n) #n, offsetof(Cell, n)
-        if (PyModule_AddObject(m, "CELL", Py_BuildValue("{sI sI sI sI sI sI sI sI sI}",
-                    OOF(ch), OOF(fg), OOF(bg), OOF(decoration_fg), OOF(cc_idx), OOF(sprite_x), OOF(sprite_y), OOF(sprite_z), "size", sizeof(Cell))) != 0) return NULL;
-#undef OOF
         PyModule_AddIntConstant(m, "BOLD", BOLD_SHIFT);
         PyModule_AddIntConstant(m, "ITALIC", ITALIC_SHIFT);
         PyModule_AddIntConstant(m, "REVERSE", REVERSE_SHIFT);
         PyModule_AddIntConstant(m, "STRIKETHROUGH", STRIKE_SHIFT);
+        PyModule_AddIntConstant(m, "DIM", DIM_SHIFT);
         PyModule_AddIntConstant(m, "DECORATION", DECORATION_SHIFT);
         PyModule_AddStringMacro(m, ERROR_PREFIX);
+#ifdef KITTY_VCS_REV
+        PyModule_AddStringMacro(m, KITTY_VCS_REV);
+#endif
         PyModule_AddIntMacro(m, CURSOR_BLOCK);
         PyModule_AddIntMacro(m, CURSOR_BEAM);
         PyModule_AddIntMacro(m, CURSOR_UNDERLINE);

kitty/data-types.h

@@ -17,6 +17,7 @@
 // Required minimum OpenGL version
 #define OPENGL_REQUIRED_VERSION_MAJOR 3
 #define OPENGL_REQUIRED_VERSION_MINOR 3
+#define GLFW_MOD_KITTY 1024
 #define UNUSED __attribute__ ((unused))
 #define PYNOARG PyObject *__a1 UNUSED, PyObject *__a2 UNUSED
 #define EXPORTED __attribute__ ((visibility ("default")))
@@ -57,6 +58,7 @@ typedef enum MouseShapes { BEAM, HAND, ARROW } MouseShape;
 #define BI_VAL(attrs) ((attrs >> 4) & 3)
 #define REVERSE_SHIFT 6
 #define STRIKE_SHIFT 7
+#define DIM_SHIFT 8
 #define COL_MASK 0xFFFFFFFF
 #define UTF8_ACCEPT 0
 #define UTF8_REJECT 1
@@ -70,11 +72,12 @@ typedef enum MouseShapes { BEAM, HAND, ARROW } MouseShape;
 
 #define CURSOR_TO_ATTRS(c, w) \
     ((w) | (((c->decoration & 3) << DECORATION_SHIFT) | ((c->bold & 1) << BOLD_SHIFT) | \
-            ((c->italic & 1) << ITALIC_SHIFT) | ((c->reverse & 1) << REVERSE_SHIFT) | ((c->strikethrough & 1) << STRIKE_SHIFT)))
+            ((c->italic & 1) << ITALIC_SHIFT) | ((c->reverse & 1) << REVERSE_SHIFT) | \
+            ((c->strikethrough & 1) << STRIKE_SHIFT) | ((c->dim & 1) << DIM_SHIFT)))
 
 #define ATTRS_TO_CURSOR(a, c) \
     (c)->decoration = (a >> DECORATION_SHIFT) & 3; (c)->bold = (a >> BOLD_SHIFT) & 1; (c)->italic = (a >> ITALIC_SHIFT) & 1; \
-    (c)->reverse = (a >> REVERSE_SHIFT) & 1; (c)->strikethrough = (a >> STRIKE_SHIFT) & 1;
+    (c)->reverse = (a >> REVERSE_SHIFT) & 1; (c)->strikethrough = (a >> STRIKE_SHIFT) & 1; (c)->dim = (a >> DIM_SHIFT) & 1;
 
 #define COPY_CELL(src, s, dest, d) \
     (dest)->cells[d] = (src)->cells[s];
@@ -160,21 +163,24 @@ typedef struct {
     Line *line;
 } LineBuf;
 
+typedef struct {
+    Cell *cells;
+    line_attrs_type *line_attrs;
+} HistoryBufSegment;
 
 typedef struct {
     PyObject_HEAD
 
-    Cell *buf;
-    index_type xnum, ynum;
+    index_type xnum, ynum, num_segments;
+    HistoryBufSegment* segments;
     Line *line;
     index_type start_of_data, count;
-    line_attrs_type *line_attrs;
 } HistoryBuf;
 
 typedef struct {
     PyObject_HEAD
 
-    bool bold, italic, reverse, strikethrough, blink;
+    bool bold, italic, reverse, strikethrough, blink, dim;
     unsigned int x, y;
     uint8_t decoration;
     CursorShape shape;
@@ -265,6 +271,5 @@ void wakeup_io_loop(bool);
 void scroll_event(double, double);
 void fake_scroll(int, bool);
 void set_special_key_combo(int glfw_key, int mods);
-void on_text_input(unsigned int codepoint, int mods);
-void on_key_input(int key, int scancode, int action, int mods);
+void on_key_input(int key, int scancode, int action, int mods, const char*, int);
 void request_window_attention(id_type, bool);

kitty/emoji.h

@@ -1,4 +1,4 @@
-// unicode data, built from the unicode standard on: 2018-02-09
+// unicode data, built from the unicode standard on: 2018-05-01
 // see gen-wcwidth.py
 #pragma once
 #include "data-types.h"

kitty/fonts.c

@@ -424,6 +424,8 @@ START_ALLOW_CASE_RANGE
         case 0x2574 ... 0x259f:
         case 0xe0b0:
         case 0xe0b2:
+        case 0xe0b4:
+        case 0xe0b6:
             return BOX_FONT;
         default:
             ans = in_symbol_maps(cell->ch);
@@ -460,6 +462,10 @@ START_ALLOW_CASE_RANGE
             return 0xfa;
         case 0xe0b2:
             return 0xfb;
+        case 0xe0b4:
+            return 0xfc;
+        case 0xe0b6:
+            return 0xfd;
         default:
             return 0xff;
     }
@@ -720,7 +726,8 @@ shape_run(Cell *first_cell, index_type num_cells, Font *font) {
             num_codepoints_used_by_glyph = UINT32_MAX;
         } else {
             next_cluster = G(info)[G(glyph_idx) + 1].cluster;
-            if (next_cluster > cluster) num_codepoints_used_by_glyph = next_cluster - cluster;
+            // RTL languages like Arabic have decreasing cluster numbers
+            if (next_cluster != cluster) num_codepoints_used_by_glyph = cluster > next_cluster ? cluster - next_cluster : next_cluster - cluster;
         }
         if (!current_group->num_glyphs) {
             add_to_current_group = true;
@@ -852,7 +859,7 @@ test_shape(PyObject UNUSED *self, PyObject *args) {
 
         PyObject *eg = PyTuple_New(MAX_NUM_EXTRA_GLYPHS);
         for (size_t g = 0; g < MAX_NUM_EXTRA_GLYPHS; g++) PyTuple_SET_ITEM(eg, g, Py_BuildValue("H", g + 1 < group->num_glyphs ? G(info)[group->first_glyph_idx + g].codepoint : 0));
-        PyList_Append(ans, Py_BuildValue("IIIN", group->num_cells, group->num_glyphs, first_glyph, eg));
+        PyList_Append(ans, Py_BuildValue("IIHN", group->num_cells, group->num_glyphs, first_glyph, eg));
         idx++;
     }
     if (face) { Py_CLEAR(face); free(font); }
@@ -897,6 +904,12 @@ render_line(Line *line) {
         ssize_t cell_font_idx = font_for_cell(cell);
         if (is_private_use(cell->ch) && i + 1 < line->xnum && (line->cells[i+1].ch == ' ' || line->cells[i+1].ch == 0) && cell_font_idx != BOX_FONT && cell_font_idx != MISSING_FONT) {
             // We have a private use char followed by a space char, render it as a two cell ligature.
+            Cell *space_cell = line->cells + i+1;
+            // Ensure the space cell uses the foreground colors from the PUA cell
+            // This is needed because there are stupid applications like
+            // powerline that use PUA+space with different foreground colors
+            // for the space and the PUA. See for example: https://github.com/kovidgoyal/kitty/issues/467
+            space_cell->fg = cell->fg; space_cell->decoration_fg = cell->decoration_fg;
             RENDER;
             render_run(line->cells + i, 2, cell_font_idx, true);
             run_font_idx = NO_FONT;
@@ -982,7 +995,7 @@ static PyObject*
 test_sprite_position_for(PyObject UNUSED *self, PyObject *args) {
     glyph_index glyph;
     ExtraGlyphs extra_glyphs = {{0}};
-    if (!PyArg_ParseTuple(args, "H|I", &glyph, &extra_glyphs.data)) return NULL;
+    if (!PyArg_ParseTuple(args, "H|H", &glyph, &extra_glyphs.data)) return NULL;
     int error;
     SpritePosition *pos = sprite_position_for(&fonts.fonts[fonts.medium_font_idx], glyph, &extra_glyphs, 0, &error);
     if (pos == NULL) { sprite_map_set_error(error); return NULL; }

kitty/fonts/box_drawing.py

@@ -3,7 +3,6 @@
 # License: GPL v3 Copyright: 2017, Kovid Goyal <kovid at kovidgoyal.net>
 
 import math
-import os
 from functools import partial as p
 from itertools import repeat
 
@@ -138,6 +137,20 @@ def cross(*s, a=1, b=1, c=1, d=1):
     half_vline(*s, level=d, which='bottom')
 
 
+def fill_region(buf, width, height, xlimits):
+    for y in range(height):
+        offset = y * width
+        for x, (upper, lower) in enumerate(xlimits):
+            buf[x + offset] = 255 if upper <= y <= lower else 0
+    # Anti-alias the boundary, simple y-axis anti-aliasing
+    for x, limits in enumerate(xlimits):
+        for y in limits:
+            for ypx in range(int(math.floor(y)), int(math.ceil(y)) + 1):
+                if 0 <= ypx < height:
+                    off = ypx * width + x
+                    buf[off] = min(255, buf[off] + int((1 - abs(y - ypx)) * 255))
+
+
 def line_equation(x1, y1, x2, y2):
     m = (y2 - y1) / (x2 - x1)
     c = y1 - m * x1
@@ -157,17 +170,91 @@ def triangle(buf, width, height, left=True):
     uppery = line_equation(x1, ay1, x2, y2)
     lowery = line_equation(x1, by1, x2, y2)
     xlimits = [(uppery(x), lowery(x)) for x in range(width)]
-    for y in range(height):
-        offset = y * width
-        for x, (upper, lower) in zip(range(width), xlimits):
-            buf[x + offset] = 255 if upper <= y <= lower else 0
-    # Anti-alias the diagonals, simple y-axis anti-aliasing
-    for x in range(width):
-        for y in xlimits[x]:
-            for ypx in range(int(math.floor(y)), int(math.ceil(y)) + 1):
-                if 0 <= ypx < height:
-                    off = ypx * width + x
-                    buf[off] = min(255, buf[off] + int((1 - abs(y - ypx)) * 255))
+    fill_region(buf, width, height, xlimits)
+
+
+def cubic_bezier(start, end, c1, c2):
+
+    def bezier_eq(p0, p1, p2, p3):
+
+        def f(t):
+            tm1 = 1 - t
+            tm1_3 = tm1 * tm1 * tm1
+            t_3 = t * t * t
+            return tm1_3 * p0 + 3 * t * tm1 * (tm1 * p1 + t * p2) + t_3 * p3
+        return f
+
+    bezier_x = bezier_eq(start[0], c1[0], c2[0], end[0])
+    bezier_y = bezier_eq(start[1], c1[1], c2[1], end[1])
+    return bezier_x, bezier_y
+
+
+def find_bezier_for_D(width, height):
+    cx = last_cx = width - 1
+    start = (0, 0)
+    end = (0, height - 1)
+    while True:
+        c1 = cx, start[1]
+        c2 = cx, end[1]
+        bezier_x, bezier_y = cubic_bezier(start, end, c1, c2)
+        if bezier_x(0.5) > width - 1:
+            return last_cx
+        last_cx = cx
+        cx += 1
+
+
+def get_bezier_limits(bezier_x, bezier_y):
+    start_x = bezier_x(0)
+    max_x = int(bezier_x(0.5))
+    last_t, t_limit = 0, 0.5
+
+    def find_t_for_x(x, start_t):
+        if abs(bezier_x(start_t) - x) < 0.1:
+            return start_t
+        increment = t_limit - start_t
+        if increment <= 0:
+            return start_t
+        while True:
+            q = bezier_x(start_t + increment)
+            if (abs(q - x) < 0.1):
+                return start_t + increment
+            if q > x:
+                increment /= 2
+                if increment < 1e-6:
+                    raise ValueError('Failed to find t for x={}'.format(x))
+            else:
+                start_t += increment
+                increment = t_limit - start_t
+                if increment <= 0:
+                    return start_t
+
+    for x in range(start_x, max_x + 1):
+        if x > start_x:
+            last_t = find_t_for_x(x, last_t)
+        upper, lower = bezier_y(last_t), bezier_y(1 - last_t)
+        if abs(upper - lower) <= 2:  # avoid pip on end of D
+            break
+        yield upper, lower
+
+
+def D(buf, width, height, left=True):
+    c1x = find_bezier_for_D(width, height)
+    start = (0, 0)
+    end = (0, height - 1)
+    c1 = c1x, start[1]
+    c2 = c1x, end[1]
+    bezier_x, bezier_y = cubic_bezier(start, end, c1, c2)
+    xlimits = list(get_bezier_limits(bezier_x, bezier_y))
+    if left:
+        fill_region(buf, width, height, xlimits)
+    else:
+        mbuf = bytearray(width * height)
+        fill_region(mbuf, width, height, xlimits)
+        for y in range(height):
+            offset = y * width
+            for src_x in range(width):
+                dest_x = width - 1 - src_x
+                buf[offset + dest_x] = mbuf[offset + src_x]
 
 
 def half_dhline(buf, width, height, level=1, which='left', only=None):
@@ -289,16 +376,42 @@ def hblock(buf, width, height, frac=1, gravity='left'):
             buf[c] = 255
 
 
-def shade(buf, width, height, frac=1/4):
-    rand = bytearray(os.urandom(width * height))
-    cutoff = int(frac * 255)
+def shade(buf, width, height, light=False, invert=False):
+    square_sz = max(1, width // 12)
+    number_of_rows = height // square_sz
+    number_of_cols = width // square_sz
+    nums = tuple(range(square_sz))
 
-    for r in range(height):
-        off = width * r
-        for c in range(width):
-            q = off + c
-            if rand[q] < cutoff:
-                buf[q] = 255
+    dest = bytearray(width * height) if invert else buf
+
+    for r in range(number_of_rows):
+        y = r * square_sz
+        is_odd = r % 2 != 0
+        if is_odd:
+            continue
+        fill_even = r % 4 == 0
+        for yr in nums:
+            y = r * square_sz + yr
+            if y >= height:
+                break
+            off = width * y
+            for c in range(number_of_cols):
+                if light:
+                    fill = (c % 4) == (0 if fill_even else 2)
+                else:
+                    fill = (c % 2 == 0) == fill_even
+                if fill:
+                    for xc in nums:
+                        x = (c * square_sz) + xc
+                        if x >= width:
+                            break
+                        dest[off + x] = 255
+    if invert:
+        for y in range(height):
+            off = width * y
+            for x in range(width):
+                q = off + x
+                buf[q] = 255 - dest[q]
 
 
 def quad(buf, width, height, x=0, y=0):
@@ -345,6 +458,8 @@ box_chars = {
     '╿': [p(half_vline, level=3), p(half_vline, which='bottom')],
     '': [triangle],
     '': [p(triangle, left=False)],
+    '': [D],
+    '': [p(D, left=False)],
     '═': [dhline],
     '║': [dvline],
     '╞': [vline, p(half_dhline, which='right')],
@@ -375,9 +490,9 @@ box_chars = {
     '▎': [p(hblock, frac=1/4)],
     '▏': [p(hblock, frac=1/8)],
     '▐': [p(hblock, frac=1/2, gravity='right')],
-    '░': [p(shade, frac=1/4)],
-    '▒': [p(shade, frac=2/4)],
-    '▓': [p(shade, frac=3/4)],
+    '░': [p(shade, light=True)],
+    '▒': [shade],
+    '▓': [p(shade, invert=True)],
     '▔': [p(vblock, frac=1/8)],
     '▕': [p(hblock, frac=1/8, gravity='right')],
     '▖': [p(quad, y=1)],
@@ -434,6 +549,26 @@ def render_missing_glyph(buf, width, height):
     draw_vline(buf, width, vgap, height - vgap + 1, width - hgap, 0)
 
 
+def test_char(ch, sz=48):
+    # kitty +runpy "from kitty.fonts.box_drawing import test_char; import sys; test_char('XXX')"
+    from .render import display_bitmap, setup_for_testing
+    from kitty.fast_data_types import concat_cells, set_send_sprite_to_gpu
+    width, height = setup_for_testing('monospace', sz)[1:]
+    buf = bytearray(width * height)
+    try:
+        render_box_char(ch, buf, width, height)
+
+        def join_cells(*cells):
+            cells = tuple(bytes(x) for x in cells)
+            return concat_cells(width, height, False, cells)
+
+        rgb_data = join_cells(buf)
+        display_bitmap(rgb_data, width, height)
+        print()
+    finally:
+        set_send_sprite_to_gpu(None)
+
+
 def test_drawing(sz=48, family='monospace'):
     from .render import display_bitmap, setup_for_testing
     from kitty.fast_data_types import concat_cells, set_send_sprite_to_gpu

kitty/fonts/render.py

@@ -220,7 +220,7 @@ def shape_string(text="abcd", family='monospace', size=11.0, dpi=96.0, path=None
 
 def display_bitmap(rgb_data, width, height):
     from tempfile import NamedTemporaryFile
-    from kitty.icat import detect_support, show
+    from kittens.icat.main import detect_support, show
     if not hasattr(display_bitmap, 'detected') and not detect_support():
         raise SystemExit('Your terminal does not support the graphics protocol')
     display_bitmap.detected = True

kitty/freetype.c

@@ -214,15 +214,20 @@ ft_face_from_path_and_psname(PyObject* path, const char* psname, void *extra_dat
 
 PyObject*
 face_from_descriptor(PyObject *descriptor) {
-#define D(key, conv) { PyObject *t = PyDict_GetItemString(descriptor, #key); if (t == NULL) return NULL; key = conv(t); t = NULL; }
-    char *path;
-    long index;
-    bool hinting;
-    long hint_style;
-    D(path, PyUnicode_AsUTF8);
-    D(index, PyLong_AsLong);
-    D(hinting, PyObject_IsTrue);
-    D(hint_style, PyLong_AsLong);
+#define D(key, conv, missing_ok) { \
+    PyObject *t = PyDict_GetItemString(descriptor, #key); \
+    if (t == NULL) { \
+        if (!missing_ok) { PyErr_SetString(PyExc_KeyError, "font descriptor is missing the key: " #key); return NULL; } \
+    } else key = conv(t); \
+}
+    char *path = NULL;
+    long index = 0;
+    bool hinting = false;
+    long hint_style = 0;
+    D(path, PyUnicode_AsUTF8, false);
+    D(index, PyLong_AsLong, true);
+    D(hinting, PyObject_IsTrue, true);
+    D(hint_style, PyLong_AsLong, true);
 #undef D
     Face *self = (Face *)Face_Type.tp_alloc(&Face_Type, 0);
     if (self != NULL) {
@@ -255,10 +260,11 @@ dealloc(Face* self) {
 
 static PyObject *
 repr(Face *self) {
+    const char *ps_name = FT_Get_Postscript_Name(self->face);
     return PyUnicode_FromFormat(
         "Face(family=%s, style=%s, ps_name=%s, path=%S, index=%d, is_scalable=%S, has_color=%S, ascender=%i, descender=%i, height=%i, underline_position=%i, underline_thickness=%i)",
         self->face->family_name ? self->face->family_name : "", self->face->style_name ? self->face->style_name : "",
-        FT_Get_Postscript_Name(self->face),
+        ps_name ? ps_name: "",
         self->path, self->index, self->is_scalable ? Py_True : Py_False, self->has_color ? Py_True : Py_False,
         self->ascender, self->descender, self->height, self->underline_position, self->underline_thickness
     );

kitty/gl-wrapper.h

@@ -1,7 +1,5 @@
 /*
-
-    OpenGL loader generated by glad 0.1.15a0 on Wed Nov  8 14:58:53 2017.
-
+    OpenGL loader generated by glad 0.1.20a0 on Fri May  4 18:11:55 2018.
     Language/Generator: C/C++ Debug
     Specification: gl
     APIs: gl=3.3
@@ -15,22 +13,17 @@
     Loader: True
     Local files: False
     Omit khrplatform: False
-
     Commandline:
         --profile="core" --api="gl=3.3" --generator="c-debug" --spec="gl" --extensions="GL_ARB_copy_image,GL_ARB_multisample,GL_ARB_robustness,GL_ARB_texture_storage,GL_KHR_debug"
     Online:
         http://glad.dav1d.de/#profile=core&language=c-debug&specification=gl&loader=on&api=gl%3D3.3&extensions=GL_ARB_copy_image&extensions=GL_ARB_multisample&extensions=GL_ARB_robustness&extensions=GL_ARB_texture_storage&extensions=GL_KHR_debug
 */
-
-
 #ifndef __glad_h_
 #define __glad_h_
-
 #ifdef __gl_h_
 #error OpenGL header already included, remove this include, glad already provides it
 #endif
 #define __gl_h_
-
 #if defined(_WIN32) && !defined(APIENTRY) && !defined(__CYGWIN__) && !defined(__SCITECH_SNAP__)
 #ifndef WIN32_LEAN_AND_MEAN
 #define WIN32_LEAN_AND_MEAN 1
@@ -40,25 +33,20 @@
 #endif
 #include <windows.h>
 #endif
-
 #ifndef APIENTRY
 #define APIENTRY
 #endif
 #ifndef APIENTRYP
 #define APIENTRYP APIENTRY *
 #endif
-
 #ifdef __cplusplus
 extern "C" {
 #endif
-
 struct gladGLversionStruct {
     int major;
     int minor;
 };
-
 typedef void* (* GLADloadproc)(const char *name);
-
 #ifndef GLAPI
 # if defined(GLAD_GLAPI_EXPORT)
 #  if defined(_WIN32) || defined(__CYGWIN__)
@@ -84,17 +72,11 @@ typedef void* (* GLADloadproc)(const char *name);
 #  define GLAPI extern
 # endif
 #endif
-
 GLAPI struct gladGLversionStruct GLVersion;
-
 GLAPI int gladLoadGL(void);
-
 GLAPI int gladLoadGLLoader(GLADloadproc);
-
-
 #define GLAD_DEBUG
 typedef void (* GLADcallback)(const char *name, void *funcptr, int len_args, ...);
-
 GLAPI void glad_set_pre_callback(GLADcallback cb);
 GLAPI void glad_set_post_callback(GLADcallback cb);
 #include <stddef.h>
@@ -1112,11 +1094,11 @@ typedef void (APIENTRYP PFNGLENABLEPROC)(GLenum cap);
 GLAPI PFNGLENABLEPROC glad_glEnable;
 GLAPI PFNGLENABLEPROC glad_debug_glEnable;
 #define glEnable glad_debug_glEnable
-typedef void (APIENTRYP PFNGLFINISHPROC)();
+typedef void (APIENTRYP PFNGLFINISHPROC)(void);
 GLAPI PFNGLFINISHPROC glad_glFinish;
 GLAPI PFNGLFINISHPROC glad_debug_glFinish;
 #define glFinish glad_debug_glFinish
-typedef void (APIENTRYP PFNGLFLUSHPROC)();
+typedef void (APIENTRYP PFNGLFLUSHPROC)(void);
 GLAPI PFNGLFLUSHPROC glad_glFlush;
 GLAPI PFNGLFLUSHPROC glad_debug_glFlush;
 #define glFlush glad_debug_glFlush
@@ -1164,7 +1146,7 @@ typedef void (APIENTRYP PFNGLGETDOUBLEVPROC)(GLenum pname, GLdouble *data);
 GLAPI PFNGLGETDOUBLEVPROC glad_glGetDoublev;
 GLAPI PFNGLGETDOUBLEVPROC glad_debug_glGetDoublev;
 #define glGetDoublev glad_debug_glGetDoublev
-typedef GLenum (APIENTRYP PFNGLGETERRORPROC)();
+typedef GLenum (APIENTRYP PFNGLGETERRORPROC)(void);
 GLAPI PFNGLGETERRORPROC glad_glGetError;
 GLAPI PFNGLGETERRORPROC glad_debug_glGetError;
 #define glGetError glad_debug_glGetError
@@ -1204,7 +1186,7 @@ typedef GLboolean (APIENTRYP PFNGLISENABLEDPROC)(GLenum cap);
 GLAPI PFNGLISENABLEDPROC glad_glIsEnabled;
 GLAPI PFNGLISENABLEDPROC glad_debug_glIsEnabled;
 #define glIsEnabled glad_debug_glIsEnabled
-typedef void (APIENTRYP PFNGLDEPTHRANGEPROC)(GLdouble near, GLdouble far);
+typedef void (APIENTRYP PFNGLDEPTHRANGEPROC)(GLdouble n, GLdouble f);
 GLAPI PFNGLDEPTHRANGEPROC glad_glDepthRange;
 GLAPI PFNGLDEPTHRANGEPROC glad_debug_glDepthRange;
 #define glDepthRange glad_debug_glDepthRange
@@ -1484,7 +1466,7 @@ typedef void (APIENTRYP PFNGLCOMPILESHADERPROC)(GLuint shader);
 GLAPI PFNGLCOMPILESHADERPROC glad_glCompileShader;
 GLAPI PFNGLCOMPILESHADERPROC glad_debug_glCompileShader;
 #define glCompileShader glad_debug_glCompileShader
-typedef GLuint (APIENTRYP PFNGLCREATEPROGRAMPROC)();
+typedef GLuint (APIENTRYP PFNGLCREATEPROGRAMPROC)(void);
 GLAPI PFNGLCREATEPROGRAMPROC glad_glCreateProgram;
 GLAPI PFNGLCREATEPROGRAMPROC glad_debug_glCreateProgram;
 #define glCreateProgram glad_debug_glCreateProgram
@@ -1884,7 +1866,7 @@ typedef void (APIENTRYP PFNGLBEGINTRANSFORMFEEDBACKPROC)(GLenum primitiveMode);
 GLAPI PFNGLBEGINTRANSFORMFEEDBACKPROC glad_glBeginTransformFeedback;
 GLAPI PFNGLBEGINTRANSFORMFEEDBACKPROC glad_debug_glBeginTransformFeedback;
 #define glBeginTransformFeedback glad_debug_glBeginTransformFeedback
-typedef void (APIENTRYP PFNGLENDTRANSFORMFEEDBACKPROC)();
+typedef void (APIENTRYP PFNGLENDTRANSFORMFEEDBACKPROC)(void);
 GLAPI PFNGLENDTRANSFORMFEEDBACKPROC glad_glEndTransformFeedback;
 GLAPI PFNGLENDTRANSFORMFEEDBACKPROC glad_debug_glEndTransformFeedback;
 #define glEndTransformFeedback glad_debug_glEndTransformFeedback
@@ -1912,7 +1894,7 @@ typedef void (APIENTRYP PFNGLBEGINCONDITIONALRENDERPROC)(GLuint id, GLenum mode)
 GLAPI PFNGLBEGINCONDITIONALRENDERPROC glad_glBeginConditionalRender;
 GLAPI PFNGLBEGINCONDITIONALRENDERPROC glad_debug_glBeginConditionalRender;
 #define glBeginConditionalRender glad_debug_glBeginConditionalRender
-typedef void (APIENTRYP PFNGLENDCONDITIONALRENDERPROC)();
+typedef void (APIENTRYP PFNGLENDCONDITIONALRENDERPROC)(void);
 GLAPI PFNGLENDCONDITIONALRENDERPROC glad_glEndConditionalRender;
 GLAPI PFNGLENDCONDITIONALRENDERPROC glad_debug_glEndConditionalRender;
 #define glEndConditionalRender glad_debug_glEndConditionalRender
@@ -2678,7 +2660,7 @@ GLAPI PFNGLSAMPLECOVERAGEARBPROC glad_debug_glSampleCoverageARB;
 #ifndef GL_ARB_robustness
 #define GL_ARB_robustness 1
 GLAPI int GLAD_GL_ARB_robustness;
-typedef GLenum (APIENTRYP PFNGLGETGRAPHICSRESETSTATUSARBPROC)();
+typedef GLenum (APIENTRYP PFNGLGETGRAPHICSRESETSTATUSARBPROC)(void);
 GLAPI PFNGLGETGRAPHICSRESETSTATUSARBPROC glad_glGetGraphicsResetStatusARB;
 GLAPI PFNGLGETGRAPHICSRESETSTATUSARBPROC glad_debug_glGetGraphicsResetStatusARB;
 #define glGetGraphicsResetStatusARB glad_debug_glGetGraphicsResetStatusARB
@@ -2798,7 +2780,7 @@ typedef void (APIENTRYP PFNGLPUSHDEBUGGROUPPROC)(GLenum source, GLuint id, GLsiz
 GLAPI PFNGLPUSHDEBUGGROUPPROC glad_glPushDebugGroup;
 GLAPI PFNGLPUSHDEBUGGROUPPROC glad_debug_glPushDebugGroup;
 #define glPushDebugGroup glad_debug_glPushDebugGroup
-typedef void (APIENTRYP PFNGLPOPDEBUGGROUPPROC)();
+typedef void (APIENTRYP PFNGLPOPDEBUGGROUPPROC)(void);
 GLAPI PFNGLPOPDEBUGGROUPPROC glad_glPopDebugGroup;
 GLAPI PFNGLPOPDEBUGGROUPPROC glad_debug_glPopDebugGroup;
 #define glPopDebugGroup glad_debug_glPopDebugGroup
@@ -2842,7 +2824,7 @@ typedef void (APIENTRYP PFNGLPUSHDEBUGGROUPKHRPROC)(GLenum source, GLuint id, GL
 GLAPI PFNGLPUSHDEBUGGROUPKHRPROC glad_glPushDebugGroupKHR;
 GLAPI PFNGLPUSHDEBUGGROUPKHRPROC glad_debug_glPushDebugGroupKHR;
 #define glPushDebugGroupKHR glad_debug_glPushDebugGroupKHR
-typedef void (APIENTRYP PFNGLPOPDEBUGGROUPKHRPROC)();
+typedef void (APIENTRYP PFNGLPOPDEBUGGROUPKHRPROC)(void);
 GLAPI PFNGLPOPDEBUGGROUPKHRPROC glad_glPopDebugGroupKHR;
 GLAPI PFNGLPOPDEBUGGROUPKHRPROC glad_debug_glPopDebugGroupKHR;
 #define glPopDebugGroupKHR glad_debug_glPopDebugGroupKHR
@@ -2867,11 +2849,8 @@ GLAPI PFNGLGETPOINTERVKHRPROC glad_glGetPointervKHR;
 GLAPI PFNGLGETPOINTERVKHRPROC glad_debug_glGetPointervKHR;
 #define glGetPointervKHR glad_debug_glGetPointervKHR
 #endif
-
 #ifdef __cplusplus
 }
 #endif
-
 #endif
-
 int init_glad(GLADloadproc, int);

kitty/gl.h

@@ -59,6 +59,7 @@ gl_init() {
         ARB_TEST(texture_storage);
 #undef ARB_TEST
         glad_loaded = true;
+        if (global_state.debug_gl) printf("GL version string: '%s' Detected version: %d.%d\n", glGetString(GL_VERSION), GLVersion.major, GLVersion.minor);
         if (GLVersion.major < OPENGL_REQUIRED_VERSION_MAJOR || (GLVersion.major == OPENGL_REQUIRED_VERSION_MAJOR && GLVersion.minor < OPENGL_REQUIRED_VERSION_MINOR)) {
             fatal("OpenGL version is %d.%d, version >= 3.3 required for kitty", GLVersion.major, GLVersion.minor);
         }

kitty/glfw-wrapper.c

@@ -254,14 +254,8 @@ load_glfw(const char* path) {
     *(void **) (&glfwSetCursor_impl) = dlsym(handle, "glfwSetCursor");
     if (glfwSetCursor_impl == NULL) fail("Failed to load glfw function glfwSetCursor with error: %s", dlerror());
 
-    *(void **) (&glfwSetKeyCallback_impl) = dlsym(handle, "glfwSetKeyCallback");
-    if (glfwSetKeyCallback_impl == NULL) fail("Failed to load glfw function glfwSetKeyCallback with error: %s", dlerror());
-
-    *(void **) (&glfwSetCharCallback_impl) = dlsym(handle, "glfwSetCharCallback");
-    if (glfwSetCharCallback_impl == NULL) fail("Failed to load glfw function glfwSetCharCallback with error: %s", dlerror());
-
-    *(void **) (&glfwSetCharModsCallback_impl) = dlsym(handle, "glfwSetCharModsCallback");
-    if (glfwSetCharModsCallback_impl == NULL) fail("Failed to load glfw function glfwSetCharModsCallback with error: %s", dlerror());
+    *(void **) (&glfwSetKeyboardCallback_impl) = dlsym(handle, "glfwSetKeyboardCallback");
+    if (glfwSetKeyboardCallback_impl == NULL) fail("Failed to load glfw function glfwSetKeyboardCallback with error: %s", dlerror());
 
     *(void **) (&glfwSetMouseButtonCallback_impl) = dlsym(handle, "glfwSetMouseButtonCallback");
     if (glfwSetMouseButtonCallback_impl == NULL) fail("Failed to load glfw function glfwSetMouseButtonCallback with error: %s", dlerror());
@@ -363,6 +357,8 @@ load_glfw(const char* path) {
 
     *(void **) (&glfwGetCocoaMonitor_impl) = dlsym(handle, "glfwGetCocoaMonitor");
 
+    *(void **) (&glfwSetCocoaTextInputFilter_impl) = dlsym(handle, "glfwSetCocoaTextInputFilter");
+
     *(void **) (&glfwGetX11Display_impl) = dlsym(handle, "glfwGetX11Display");
 
     *(void **) (&glfwGetX11Window_impl) = dlsym(handle, "glfwGetX11Window");

kitty/glfw-wrapper.h

@@ -1,6 +1,7 @@
 #pragma once
 #include <stddef.h>
 #include <stdint.h>
+typedef int (* GLFWcocoatextinputfilterfun)(int,int,unsigned int);
 
 
 /*! @name GLFW version macros
@@ -1133,9 +1134,15 @@ typedef void (* GLFWcursorenterfun)(GLFWwindow*,int);
  */
 typedef void (* GLFWscrollfun)(GLFWwindow*,double,double);
 
-/*! @brief The function signature for keyboard key callbacks.
+/*! @brief The function signature for key callbacks.
  *
- *  This is the function signature for keyboard key callback functions.
+ *  This is the function signature for key callback functions.
+ *  The semantics of this function are that the key that is interacted with on the
+ *  keyboard is reported, and the text, if any generated by the key is reported.
+ *  So, for example, if on a US-ASCII keyboard the user presses Shift+= GLFW
+ *  will report the text "+" and the key as GLFW_KEY_EQUAL. The reported key takes into
+ *  account any current keyboard maps defined in the OS. So with a dvorak mapping, pressing
+ *  the "s" key will generate text "o" and GLFW_KEY_O.
  *
  *  @param[in] window The window that received the event.
  *  @param[in] key The [keyboard key](@ref keys) that was pressed or released.
@@ -1143,56 +1150,22 @@ typedef void (* GLFWscrollfun)(GLFWwindow*,double,double);
  *  @param[in] action `GLFW_PRESS`, `GLFW_RELEASE` or `GLFW_REPEAT`.
  *  @param[in] mods Bit field describing which [modifier keys](@ref mods) were
  *  held down.
+ *  @param[in] text UTF-8 encoded text generated by this key event or empty string.
+ *  @param[in] reserved Reserved for future use.
+ *
+ *  @note On X11/Wayland if a modifier other than the modifiers GLFW reports
+ *  (ctrl/shift/alt/super) is used, GLFW will report the shifted key rather
+ *  than the unshifted key. So for example, if ISO_Shift_Level_5 is used to
+ *  convert the key A into UP GLFW will report the key as UP with no modifiers.
  *
  *  @sa @ref input_key
- *  @sa @ref glfwSetKeyCallback
+ *  @sa @ref glfwSetKeyboardCallback
  *
- *  @since Added in version 1.0.
- *  @glfw3 Added window handle, scancode and modifier mask parameters.
+ *  @since Added in version 4.0.
  *
  *  @ingroup input
  */
-typedef void (* GLFWkeyfun)(GLFWwindow*,int,int,int,int);
-
-/*! @brief The function signature for Unicode character callbacks.
- *
- *  This is the function signature for Unicode character callback functions.
- *
- *  @param[in] window The window that received the event.
- *  @param[in] codepoint The Unicode code point of the character.
- *
- *  @sa @ref input_char
- *  @sa @ref glfwSetCharCallback
- *
- *  @since Added in version 2.4.
- *  @glfw3 Added window handle parameter.
- *
- *  @ingroup input
- */
-typedef void (* GLFWcharfun)(GLFWwindow*,unsigned int);
-
-/*! @brief The function signature for Unicode character with modifiers
- *  callbacks.
- *
- *  This is the function signature for Unicode character with modifiers callback
- *  functions.  It is called for each input character, regardless of what
- *  modifier keys are held down.
- *
- *  @param[in] window The window that received the event.
- *  @param[in] codepoint The Unicode code point of the character.
- *  @param[in] mods Bit field describing which [modifier keys](@ref mods) were
- *  held down.
- *
- *  @sa @ref input_char
- *  @sa @ref glfwSetCharModsCallback
- *
- *  @deprecated Scheduled for removal in version 4.0.
- *
- *  @since Added in version 3.1.
- *
- *  @ingroup input
- */
-typedef void (* GLFWcharmodsfun)(GLFWwindow*,unsigned int,int);
+typedef void (* GLFWkeyboardfun)(GLFWwindow*, int, int, int, int, const char*, int);
 
 /*! @brief The function signature for file drop callbacks.
  *
@@ -1713,17 +1686,9 @@ typedef void (*glfwSetCursor_func)(GLFWwindow*, GLFWcursor*);
 glfwSetCursor_func glfwSetCursor_impl;
 #define glfwSetCursor glfwSetCursor_impl
 
-typedef GLFWkeyfun (*glfwSetKeyCallback_func)(GLFWwindow*, GLFWkeyfun);
-glfwSetKeyCallback_func glfwSetKeyCallback_impl;
-#define glfwSetKeyCallback glfwSetKeyCallback_impl
-
-typedef GLFWcharfun (*glfwSetCharCallback_func)(GLFWwindow*, GLFWcharfun);
-glfwSetCharCallback_func glfwSetCharCallback_impl;
-#define glfwSetCharCallback glfwSetCharCallback_impl
-
-typedef GLFWcharmodsfun (*glfwSetCharModsCallback_func)(GLFWwindow*, GLFWcharmodsfun);
-glfwSetCharModsCallback_func glfwSetCharModsCallback_impl;
-#define glfwSetCharModsCallback glfwSetCharModsCallback_impl
+typedef GLFWkeyboardfun (*glfwSetKeyboardCallback_func)(GLFWwindow*, GLFWkeyboardfun);
+glfwSetKeyboardCallback_func glfwSetKeyboardCallback_impl;
+#define glfwSetKeyboardCallback glfwSetKeyboardCallback_impl
 
 typedef GLFWmousebuttonfun (*glfwSetMouseButtonCallback_func)(GLFWwindow*, GLFWmousebuttonfun);
 glfwSetMouseButtonCallback_func glfwSetMouseButtonCallback_impl;
@@ -1861,6 +1826,10 @@ typedef uint32_t (*glfwGetCocoaMonitor_func)(GLFWmonitor*);
 glfwGetCocoaMonitor_func glfwGetCocoaMonitor_impl;
 #define glfwGetCocoaMonitor glfwGetCocoaMonitor_impl
 
+typedef GLFWcocoatextinputfilterfun (*glfwSetCocoaTextInputFilter_func)(GLFWwindow*, GLFWcocoatextinputfilterfun);
+glfwSetCocoaTextInputFilter_func glfwSetCocoaTextInputFilter_impl;
+#define glfwSetCocoaTextInputFilter glfwSetCocoaTextInputFilter_impl
+
 typedef void* (*glfwGetX11Display_func)();
 glfwGetX11Display_func glfwGetX11Display_impl;
 #define glfwGetX11Display glfwGetX11Display_impl

kitty/glfw.c

@@ -9,13 +9,15 @@
 #include "glfw-wrapper.h"
 extern bool cocoa_make_window_resizable(void *w);
 extern void cocoa_create_global_menu(void);
-extern void cocoa_set_titlebar_color(void *w);
+extern void cocoa_set_hide_from_tasks(void);
+extern void cocoa_set_titlebar_color(void *w, color_type color);
 
 #if GLFW_KEY_LAST >= MAX_KEY_COUNT
 #error "glfw has too many keys, you should increase MAX_KEY_COUNT"
 #endif
 
 static GLFWcursor *standard_cursor = NULL, *click_cursor = NULL, *arrow_cursor = NULL;
+static bool event_loop_blocking_with_no_timeout = false;
 
 void
 update_os_window_viewport(OSWindow *window, bool notify_boss) {
@@ -74,9 +76,20 @@ is_window_ready_for_callbacks() {
 
 static inline void
 show_mouse_cursor(GLFWwindow *w) {
-    if (glfwGetInputMode(w, GLFW_CURSOR) != GLFW_CURSOR_NORMAL) { glfwSetInputMode(w, GLFW_CURSOR, GLFW_CURSOR_NORMAL); }
+    glfwSetInputMode(w, GLFW_CURSOR, GLFW_CURSOR_NORMAL);
 }
 
+// On Cocoa, glfwWaitEvents() can block indefinitely because of the way Cocoa
+// works. See https://github.com/glfw/glfw/issues/1251. I have noticed this
+// happening in particular with window resize events, when waiting with no
+// timeout. See https://github.com/kovidgoyal/kitty/issues/458
+// So we use an unlovely hack to workaround that case
+#ifdef __APPLE__
+#define unjam_event_loop() { if (event_loop_blocking_with_no_timeout) wakeup_main_loop(); }
+#else
+#define unjam_event_loop()
+#endif
+
 static void
 framebuffer_size_callback(GLFWwindow *w, int width, int height) {
     if (!set_callback_window(w)) return;
@@ -84,6 +97,7 @@ framebuffer_size_callback(GLFWwindow *w, int width, int height) {
         OSWindow *window = global_state.callback_os_window;
         window->has_pending_resizes = true; global_state.has_pending_resizes = true;
         window->last_resize_event_at = monotonic();
+        unjam_event_loop();
     } else log_error("Ignoring resize request for tiny size: %dx%d", width, height);
     global_state.callback_os_window = NULL;
 }
@@ -96,21 +110,13 @@ refresh_callback(GLFWwindow *w) {
 }
 
 static void
-char_mods_callback(GLFWwindow *w, unsigned int codepoint, int mods) {
-    if (!set_callback_window(w)) return;
-    global_state.callback_os_window->cursor_blink_zero_time = monotonic();
-    if (is_window_ready_for_callbacks()) on_text_input(codepoint, mods);
-    global_state.callback_os_window = NULL;
-}
-
-static void
-key_callback(GLFWwindow *w, int key, int scancode, int action, int mods) {
+key_callback(GLFWwindow *w, int key, int scancode, int action, int mods, const char* text, int state) {
     if (!set_callback_window(w)) return;
     global_state.callback_os_window->cursor_blink_zero_time = monotonic();
     if (key >= 0 && key <= GLFW_KEY_LAST) {
         global_state.callback_os_window->is_key_pressed[key] = action == GLFW_RELEASE ? false : true;
-        if (is_window_ready_for_callbacks()) on_key_input(key, scancode, action, mods);
     }
+    if (is_window_ready_for_callbacks()) on_key_input(key, scancode, action, mods, text, state);
     global_state.callback_os_window = NULL;
 }
 
@@ -231,7 +237,11 @@ static GLFWimage logo = {0};
 static PyObject*
 set_default_window_icon(PyObject UNUSED *self, PyObject *args) {
     Py_ssize_t sz;
-    if(!PyArg_ParseTuple(args, "s#ii", &(logo.pixels), &sz, &(logo.width), &(logo.height))) return NULL;
+    const char *logo_data;
+    if(!PyArg_ParseTuple(args, "s#ii", &(logo_data), &sz, &(logo.width), &(logo.height))) return NULL;
+    sz = (MAX(logo.width * logo.height, sz));
+    logo.pixels = malloc(sz);
+    if (logo.pixels) memcpy(logo.pixels, logo_data, sz);
     Py_RETURN_NONE;
 }
 
@@ -300,12 +310,29 @@ set_dpi_from_os_window(OSWindow *w) {
 
 static bool is_first_window = true;
 
+#ifdef __APPLE__
+static int
+filter_option(int key UNUSED, int mods, unsigned int scancode UNUSED) {
+    return ((mods == GLFW_MOD_ALT) || (mods == (GLFW_MOD_ALT | GLFW_MOD_SHIFT))) ? 1 : 0;
+}
+#endif
+
+void
+set_titlebar_color(OSWindow *w, color_type color) {
+    if (w->handle && (!w->last_titlebar_color || (w->last_titlebar_color & 0xffffff) != (color & 0xffffff))) {
+        w->last_titlebar_color = (1 << 24) | (color & 0xffffff);
+#ifdef __APPLE__
+        cocoa_set_titlebar_color(glfwGetCocoaWindow(w->handle), color);
+#endif
+    }
+}
+
 static PyObject*
 create_os_window(PyObject UNUSED *self, PyObject *args) {
     int width, height, x = -1, y = -1;
     char *title, *wm_class_class, *wm_class_name;
     PyObject *load_programs = NULL;
-    if (!PyArg_ParseTuple(args, "iisss|Oiii", &width, &height, &title, &wm_class_name, &wm_class_class, &load_programs, &x, &y)) return NULL;
+    if (!PyArg_ParseTuple(args, "iisss|Oii", &width, &height, &title, &wm_class_name, &wm_class_class, &load_programs, &x, &y)) return NULL;
 
     if (is_first_window) {
         glfwWindowHint(GLFW_VISIBLE, GLFW_FALSE);
@@ -338,7 +365,7 @@ create_os_window(PyObject UNUSED *self, PyObject *args) {
         PyErr_SetString(PyExc_ValueError, "Too many windows");
         return NULL;
     }
-    bool want_semi_transparent = (1.0 - OPT(background_opacity) > 0.1) ? true : false;
+    bool want_semi_transparent = (1.0 - OPT(background_opacity) >= 0.01) ? true : false;
     glfwWindowHint(GLFW_TRANSPARENT_FRAMEBUFFER, want_semi_transparent);
     GLFWwindow *glfw_window = glfwCreateWindow(width, height, title, NULL, global_state.num_os_windows ? global_state.os_windows[0].handle : NULL);
     if (glfw_window == NULL) {
@@ -358,6 +385,9 @@ create_os_window(PyObject UNUSED *self, PyObject *args) {
         Py_DECREF(ret);
 #ifdef __APPLE__
         cocoa_create_global_menu();
+        if (OPT(macos_option_as_alt)) glfwSetCocoaTextInputFilter(glfw_window, filter_option);
+        // This needs to be done only after the first window has been created, because glfw only sets the activation policy once upon initialization.
+        if (OPT(macos_hide_from_tasks)) cocoa_set_hide_from_tasks();
 #endif
         is_first_window = false;
     }
@@ -375,11 +405,10 @@ create_os_window(PyObject UNUSED *self, PyObject *args) {
     update_os_window_viewport(w, false);
     glfwSetFramebufferSizeCallback(glfw_window, framebuffer_size_callback);
     glfwSetWindowRefreshCallback(glfw_window, refresh_callback);
-    glfwSetCharModsCallback(glfw_window, char_mods_callback);
     glfwSetMouseButtonCallback(glfw_window, mouse_button_callback);
     glfwSetScrollCallback(glfw_window, scroll_callback);
     glfwSetCursorPosCallback(glfw_window, cursor_pos_callback);
-    glfwSetKeyCallback(glfw_window, key_callback);
+    glfwSetKeyboardCallback(glfw_window, key_callback);
     glfwSetWindowFocusCallback(glfw_window, window_focus_callback);
     glfwSetDropCallback(glfw_window, drop_callback);
 #ifdef __APPLE__
@@ -387,7 +416,6 @@ create_os_window(PyObject UNUSED *self, PyObject *args) {
         if (glfwGetCocoaWindow) { if (!cocoa_make_window_resizable(glfwGetCocoaWindow(glfw_window))) { PyErr_Print(); } }
         else log_error("Failed to load glfwGetCocoaWindow");
     }
-    cocoa_set_titlebar_color(glfwGetCocoaWindow(glfw_window));
 #endif
     double now = monotonic();
     w->is_focused = true;
@@ -488,18 +516,6 @@ glfw_terminate(PYNOARG) {
     Py_RETURN_NONE;
 }
 
-PyObject*
-glfw_wait_events(PyObject UNUSED *self, PyObject *args) {
-    double time = -1;
-    if (PyTuple_GET_SIZE(args) > 0) {
-        time = PyFloat_AsDouble(PyTuple_GET_ITEM(args, 0));
-        if (PyErr_Occurred()) PyErr_Clear();
-    }
-    if (time < 0) glfwWaitEvents();
-    else glfwWaitEventsTimeout(time);
-    Py_RETURN_NONE;
-}
-
 PyObject*
 glfw_post_empty_event(PYNOARG) {
     glfwPostEmptyEvent();
@@ -613,7 +629,7 @@ request_window_attention(id_type kitty_window_id, bool audio_bell) {
     OSWindow *w = os_window_for_kitty_window(kitty_window_id);
     if (w) {
         if (audio_bell) ring_audio_bell(w);
-        glfwRequestWindowAttention(w->handle);
+        if (OPT(window_alert_on_bell)) glfwRequestWindowAttention(w->handle);
         glfwPostEmptyEvent();
     }
 }
@@ -625,9 +641,7 @@ set_os_window_title(OSWindow *w, const char *title) {
 
 void
 hide_mouse(OSWindow *w) {
-    if (glfwGetInputMode(w->handle, GLFW_CURSOR) != GLFW_CURSOR_HIDDEN) {
-        glfwSetInputMode(w->handle, GLFW_CURSOR, GLFW_CURSOR_HIDDEN);
-    }
+    glfwSetInputMode(w->handle, GLFW_CURSOR, GLFW_CURSOR_HIDDEN);
 }
 
 void
@@ -637,8 +651,8 @@ swap_window_buffers(OSWindow *w) {
 
 void
 event_loop_wait(double timeout) {
-    if (timeout < 0) glfwWaitEvents();
-    else if (timeout > 0) glfwWaitEventsTimeout(timeout);
+    if (timeout < 0) { event_loop_blocking_with_no_timeout = true; glfwWaitEvents(); event_loop_blocking_with_no_timeout = false; }
+    else glfwWaitEventsTimeout(timeout);
 }
 
 void
@@ -751,6 +765,16 @@ os_window_swap_buffers(PyObject UNUSED *self, PyObject *args) {
     return NULL;
 }
 
+static PyObject*
+ring_bell(PyObject UNUSED *self, PyObject *args) {
+    id_type os_window_id;
+    if (!PyArg_ParseTuple(args, "K", &os_window_id)) return NULL;
+    OSWindow *w = os_window_for_kitty_window(os_window_id);
+    if (w && w->handle) {
+        glfwWindowBell(w->handle);
+    }
+    Py_RETURN_NONE;
+}
 // Boilerplate {{{
 
 static PyMethodDef module_methods[] = {
@@ -764,6 +788,7 @@ static PyMethodDef module_methods[] = {
     METHODB(glfw_window_hint, METH_VARARGS),
     METHODB(os_window_should_close, METH_VARARGS),
     METHODB(os_window_swap_buffers, METH_VARARGS),
+    METHODB(ring_bell, METH_VARARGS),
     METHODB(get_primary_selection, METH_NOARGS),
     METHODB(x11_display, METH_NOARGS),
     METHODB(x11_window_id, METH_O),
@@ -771,7 +796,6 @@ static PyMethodDef module_methods[] = {
     METHODB(glfw_poll_events, METH_NOARGS),
     {"glfw_init", (PyCFunction)glfw_init, METH_VARARGS, ""},
     {"glfw_terminate", (PyCFunction)glfw_terminate, METH_NOARGS, ""},
-    {"glfw_wait_events", (PyCFunction)glfw_wait_events, METH_VARARGS, ""},
     {"glfw_post_empty_event", (PyCFunction)glfw_post_empty_event, METH_NOARGS, ""},
     {"glfw_get_physical_dpi", (PyCFunction)glfw_get_physical_dpi, METH_NOARGS, ""},
     {"glfw_get_key_name", (PyCFunction)glfw_get_key_name, METH_VARARGS, ""},
@@ -780,10 +804,19 @@ static PyMethodDef module_methods[] = {
     {NULL, NULL, 0, NULL}        /* Sentinel */
 };
 
+void cleanup_glfw(void) {
+    if (logo.pixels) free(logo.pixels);
+    logo.pixels = NULL;
+}
+
 // constants {{{
 bool
 init_glfw(PyObject *m) {
     if (PyModule_AddFunctions(m, module_methods) != 0) return false;
+    if (Py_AtExit(cleanup_glfw) != 0) {
+        PyErr_SetString(PyExc_RuntimeError, "Failed to register the glfw exit handler");
+        return false;
+    }
 #define ADDC(n) if(PyModule_AddIntConstant(m, #n, n) != 0) return false;
     ADDC(GLFW_RELEASE);
     ADDC(GLFW_PRESS);
@@ -925,6 +958,7 @@ init_glfw(PyObject *m) {
     ADDC(GLFW_MOD_CONTROL);
     ADDC(GLFW_MOD_ALT);
     ADDC(GLFW_MOD_SUPER);
+    ADDC(GLFW_MOD_KITTY);
 
 // --- Mouse -------------------------------------------------------------------
     ADDC(GLFW_MOUSE_BUTTON_1);

kitty/graphics.c

@@ -873,9 +873,10 @@ new(PyTypeObject UNUSED *type, PyObject UNUSED *args, PyObject UNUSED *kwds) {
 static inline PyObject*
 image_as_dict(Image *img) {
 #define U(x) #x, img->x
-    return Py_BuildValue("{sI sI sI sI sI sI sH sH sN}",
-        U(texture_id), U(client_id), U(width), U(height), U(internal_id), U(refcnt), U(data_loaded),
-        "is_4byte_aligned", img->load_data.is_4byte_aligned,
+    return Py_BuildValue("{sI sI sI sI sI sI sO sO sN}",
+        U(texture_id), U(client_id), U(width), U(height), U(internal_id), U(refcnt),
+        "data_loaded", img->data_loaded ? Py_True : Py_False,
+        "is_4byte_aligned", img->load_data.is_4byte_aligned ? Py_True : Py_False,
         "data", Py_BuildValue("y#", img->load_data.data, img->load_data.data_sz)
     );
 #undef U

kitty/history.c

@@ -10,10 +10,42 @@
 #include <structmember.h>
 
 extern PyTypeObject Line_Type;
+#define SEGMENT_SIZE 2048
+
+static inline void
+add_segment(HistoryBuf *self) {
+    self->num_segments += 1;
+    self->segments = PyMem_Realloc(self->segments, sizeof(HistoryBufSegment) * self->num_segments);
+    if (self->segments == NULL) fatal("Out of memory allocating new history buffer segment");
+    HistoryBufSegment *s = self->segments + self->num_segments - 1;
+    s->cells = PyMem_Calloc(self->xnum * SEGMENT_SIZE, sizeof(Cell));
+    s->line_attrs = PyMem_Calloc(SEGMENT_SIZE, sizeof(line_attrs_type));
+    if (s->cells == NULL || s->line_attrs == NULL) fatal("Out of memory allocating new history buffer segment");
+}
+
+static inline index_type
+segment_for(HistoryBuf *self, index_type y) {
+    index_type seg_num = y / SEGMENT_SIZE;
+    while (UNLIKELY(seg_num >= self->num_segments && SEGMENT_SIZE * self->num_segments < self->ynum)) add_segment(self);
+    if (UNLIKELY(seg_num >= self->num_segments)) fatal("Out of bounds access to history buffer line number: %u", y);
+    return seg_num;
+}
+
+#define seg_ptr(which, stride) { \
+    index_type seg_num = segment_for(self, y); \
+    y -= seg_num * SEGMENT_SIZE; \
+    return self->segments[seg_num].which + y * stride; \
+}
 
 static inline Cell*
-lineptr(HistoryBuf *linebuf, index_type y) {
-    return linebuf->buf + y * linebuf->xnum;
+lineptr(HistoryBuf *self, index_type y) {
+    seg_ptr(cells, self->xnum);
+}
+
+
+static inline line_attrs_type*
+attrptr(HistoryBuf *self, index_type y) {
+    seg_ptr(line_attrs, 1);
 }
 
 static PyObject *
@@ -32,18 +64,12 @@ new(PyTypeObject *type, PyObject *args, PyObject UNUSED *kwds) {
     if (self != NULL) {
         self->xnum = xnum;
         self->ynum = ynum;
-        self->buf = PyMem_Calloc(xnum * ynum, sizeof(Cell));
-        self->line_attrs = PyMem_Calloc(ynum, sizeof(line_attrs_type));
         self->line = alloc_line();
-        if (self->buf == NULL || self->line == NULL || self->line_attrs == NULL) {
-            PyErr_NoMemory();
-            PyMem_Free(self->buf); Py_CLEAR(self->line); PyMem_Free(self->line_attrs);
-            Py_CLEAR(self);
-        } else {
-            self->line->xnum = xnum;
-            for(index_type y = 0; y < self->ynum; y++) {
-                clear_chars_in_line(lineptr(self, y), self->xnum, BLANK_CHAR);
-            }
+        self->num_segments = 0;
+        add_segment(self);
+        self->line->xnum = xnum;
+        for(index_type y = 0; y < self->ynum; y++) {
+            clear_chars_in_line(lineptr(self, y), self->xnum, BLANK_CHAR);
         }
     }
 
@@ -53,8 +79,11 @@ new(PyTypeObject *type, PyObject *args, PyObject UNUSED *kwds) {
 static void
 dealloc(HistoryBuf* self) {
     Py_CLEAR(self->line);
-    PyMem_Free(self->buf);
-    PyMem_Free(self->line_attrs);
+    for (size_t i = 0; i < self->num_segments; i++) {
+        PyMem_Free(self->segments[i].cells);
+        PyMem_Free(self->segments[i].line_attrs);
+    }
+    PyMem_Free(self->segments);
     Py_TYPE(self)->tp_free((PyObject*)self);
 }
 
@@ -71,8 +100,8 @@ static inline void
 init_line(HistoryBuf *self, index_type num, Line *l) {
     // Initialize the line l, setting its pointer to the offsets for the line at index (buffer position) num
     l->cells = lineptr(self, num);
-    l->continued = self->line_attrs[num] & CONTINUED_MASK;
-    l->has_dirty_text = self->line_attrs[num] & TEXT_DIRTY_MASK ? true : false;
+    l->continued = *attrptr(self, num) & CONTINUED_MASK;
+    l->has_dirty_text = *attrptr(self, num) & TEXT_DIRTY_MASK ? true : false;
 }
 
 void
@@ -82,12 +111,14 @@ historybuf_init_line(HistoryBuf *self, index_type lnum, Line *l) {
 
 void
 historybuf_mark_line_clean(HistoryBuf *self, index_type y) {
-    self->line_attrs[index_of(self, y)] &= ~TEXT_DIRTY_MASK;
+    line_attrs_type *p = attrptr(self, index_of(self, y));
+    *p &= ~TEXT_DIRTY_MASK;
 }
 
 void
 historybuf_mark_line_dirty(HistoryBuf *self, index_type y) {
-    self->line_attrs[index_of(self, y)] |= TEXT_DIRTY_MASK;
+    line_attrs_type *p = attrptr(self, index_of(self, y));
+    *p |= TEXT_DIRTY_MASK;
 }
 
 inline void
@@ -105,43 +136,11 @@ historybuf_push(HistoryBuf *self) {
     return idx;
 }
 
-bool
-historybuf_resize(HistoryBuf *self, index_type lines) {
-    HistoryBuf t = {{0}};
-    t.xnum=self->xnum;
-    t.ynum=lines;
-    if (t.ynum > 0 && t.ynum != self->ynum) {
-        t.buf = PyMem_Calloc(t.xnum * t.ynum, sizeof(Cell));
-        if (t.buf == NULL) { PyErr_NoMemory(); return false; }
-        t.line_attrs = PyMem_Calloc(t.ynum, sizeof(line_attrs_type));
-        if (t.line_attrs == NULL) { PyMem_Free(t.buf); PyErr_NoMemory(); return false; }
-        t.count = MIN(self->count, t.ynum);
-        for (index_type s=0; s < t.count; s++) {
-            index_type si = index_of(self, s), ti = index_of(&t, s);
-            copy_cells(lineptr(self, si), lineptr(&t, ti), t.xnum);
-            t.line_attrs[ti] = self->line_attrs[si];
-        }
-        self->count = t.count;
-        self->start_of_data = t.start_of_data;
-        self->ynum = t.ynum;
-        PyMem_Free(self->buf); PyMem_Free(self->line_attrs);
-        self->buf = t.buf; self->line_attrs = t.line_attrs;
-    }
-    return true;
-}
-
 void
 historybuf_add_line(HistoryBuf *self, const Line *line) {
     index_type idx = historybuf_push(self);
     copy_line(line, self->line);
-    self->line_attrs[idx] = (line->continued & CONTINUED_MASK) | (line->has_dirty_text ? TEXT_DIRTY_MASK : 0);
-}
-
-static PyObject*
-change_num_of_lines(HistoryBuf *self, PyObject *val) {
-#define change_num_of_lines_doc "Change the number of lines in this buffer"
-    if(!historybuf_resize(self, (index_type)PyLong_AsUnsignedLong(val))) return NULL;
-    Py_RETURN_NONE;
+    *attrptr(self, idx) = (line->continued & CONTINUED_MASK) | (line->has_dirty_text ? TEXT_DIRTY_MASK : 0);
 }
 
 static PyObject*
@@ -188,7 +187,7 @@ as_ansi(HistoryBuf *self, PyObject *callback) {
     for(unsigned int i = 0; i < self->count; i++) {
         init_line(self, i, &l);
         if (i < self->count - 1) {
-            l.continued = self->line_attrs[index_of(self, i + 1)] & CONTINUED_MASK;
+            l.continued = *attrptr(self, index_of(self, i + 1)) & CONTINUED_MASK;
         } else l.continued = false;
         index_type num = line_as_ansi(&l, t, 5120);
         if (!(l.continued) && num < 5119) t[num++] = 10; // 10 = \n
@@ -209,7 +208,7 @@ static PyObject*
 as_text(HistoryBuf *self, PyObject *args) {
     Line l = {.xnum=self->xnum};
 #define gl(self, y) get_line(self, y, &l);
-    as_text_generic(args, self, gl, self->count, self->xnum, callback, as_ansi);
+    as_text_generic(args, self, gl, self->count, self->xnum);
 #undef gl
 }
 
@@ -219,7 +218,7 @@ dirty_lines(HistoryBuf *self, PyObject *a UNUSED) {
 #define dirty_lines_doc "dirty_lines() -> Line numbers of all lines that have dirty text."
     PyObject *ans = PyList_New(0);
     for (index_type i = 0; i < self->ynum; i++) {
-        if (self->line_attrs[i] & TEXT_DIRTY_MASK) {
+        if (*attrptr(self, i) & TEXT_DIRTY_MASK) {
             PyList_Append(ans, PyLong_FromUnsignedLong(i));
         }
     }
@@ -232,7 +231,6 @@ static PyObject* rewrap(HistoryBuf *self, PyObject *args);
 #define rewrap_doc ""
 
 static PyMethodDef methods[] = {
-    METHOD(change_num_of_lines, METH_O)
     METHOD(line, METH_O)
     METHOD(as_ansi, METH_O)
     METHODB(as_text, METH_VARARGS),
@@ -275,19 +273,22 @@ HistoryBuf *alloc_historybuf(unsigned int lines, unsigned int columns) {
 
 #define init_src_line(src_y) init_line(src, map_src_index(src_y), src->line);
 
-#define is_src_line_continued(src_y) (map_src_index(src_y) < src->ynum - 1 ? (src->line_attrs[map_src_index(src_y + 1)] & CONTINUED_MASK) : false)
+#define is_src_line_continued(src_y) (map_src_index(src_y) < src->ynum - 1 ? (*attrptr(src, map_src_index(src_y + 1)) & CONTINUED_MASK) : false)
 
-#define next_dest_line(cont) dest->line_attrs[historybuf_push(dest)] = cont & CONTINUED_MASK; dest->line->continued = cont;
+#define next_dest_line(cont) *attrptr(dest, historybuf_push(dest)) = cont & CONTINUED_MASK; dest->line->continued = cont;
 
 #define first_dest_line next_dest_line(false);
 
 #include "rewrap.h"
 
 void historybuf_rewrap(HistoryBuf *self, HistoryBuf *other) {
-    // Fast path
+    while(other->num_segments < self->num_segments) add_segment(other);
     if (other->xnum == self->xnum && other->ynum == self->ynum) {
-        memcpy(other->buf, self->buf, sizeof(Cell) * self->xnum * self->ynum);
-        memcpy(other->line_attrs, self->line_attrs, sizeof(line_attrs_type) * self->ynum);
+        // Fast path
+        for (index_type i = 0; i < self->num_segments; i++) {
+            memcpy(other->segments[i].cells, self->segments[i].cells, SEGMENT_SIZE * self->xnum * sizeof(Cell));
+            memcpy(other->segments[i].line_attrs, self->segments[i].line_attrs, SEGMENT_SIZE * sizeof(line_attrs_type));
+        }
         other->count = self->count; other->start_of_data = self->start_of_data;
         return;
     }
@@ -295,7 +296,7 @@ void historybuf_rewrap(HistoryBuf *self, HistoryBuf *other) {
     index_type x = 0, y = 0;
     if (self->count > 0) {
         rewrap_inner(self, other, self->count, NULL, &x, &y);
-        for (index_type i = 0; i < other->count; i++) other->line_attrs[(other->start_of_data + i) % other->ynum] |= TEXT_DIRTY_MASK;
+        for (index_type i = 0; i < other->count; i++) *attrptr(other, (other->start_of_data + i) % other->ynum) |= TEXT_DIRTY_MASK;
     }
 }
 

kitty/keys.c

@@ -43,22 +43,6 @@ active_window() {
     return NULL;
 }
 
-void
-on_text_input(unsigned int codepoint, int mods) {
-    Window *w = active_window();
-    static char buf[16];
-    unsigned int sz = 0;
-
-    if (w != NULL) {
-        bool is_text = mods <= GLFW_MOD_SHIFT;
-        if (is_text) sz = encode_utf8(codepoint, buf);
-#ifdef __APPLE__
-        if (!OPT(macos_option_as_alt) && IS_ALT_MODS(mods)) sz = encode_utf8(codepoint, buf);
-#endif
-        if (sz) schedule_write_to_child(w->id, buf, sz);
-    }
-}
-
 static inline bool
 is_modifier_key(int key) {
     switch(key) {
@@ -76,66 +60,6 @@ is_modifier_key(int key) {
     }
 }
 
-static inline int
-get_localized_key(int key, int scancode) {
-    const char *name = glfwGetKeyName(key, scancode);
-    if (name == NULL || name[1] != 0) return key;
-    switch(name[0]) {
-#define K(ch, name) case ch: return GLFW_KEY_##name
-        // key names {{{
-        K('A', A); K('a', A);
-        K('B', B); K('b', B);
-        K('C', C); K('c', C);
-        K('D', D); K('d', D);
-        K('E', E); K('e', E);
-        K('F', F); K('f', F);
-        K('G', G); K('g', G);
-        K('H', H); K('h', H);
-        K('I', I); K('i', I);
-        K('J', J); K('j', J);
-        K('K', K); K('k', K);
-        K('L', L); K('l', L);
-        K('M', M); K('m', M);
-        K('N', N); K('n', N);
-        K('O', O); K('o', O);
-        K('P', P); K('p', P);
-        K('Q', Q); K('q', Q);
-        K('R', R); K('r', R);
-        K('S', S); K('s', S);
-        K('T', T); K('t', T);
-        K('U', U); K('u', U);
-        K('V', V); K('v', V);
-        K('W', W); K('w', W);
-        K('X', X); K('x', X);
-        K('Y', Y); K('y', Y);
-        K('Z', Z); K('z', Z);
-        K('0', 0);
-        K('1', 1);
-        K('2', 2);
-        K('3', 3);
-        K('5', 5);
-        K('6', 6);
-        K('7', 7);
-        K('8', 8);
-        K('9', 9);
-        K('\'', APOSTROPHE);
-        K(',', COMMA);
-        K('.', PERIOD);
-        K('/', SLASH);
-        K('-', MINUS);
-        K(';', SEMICOLON);
-        K('=', EQUAL);
-        K('[', LEFT_BRACKET);
-        K(']', RIGHT_BRACKET);
-        K('`', GRAVE_ACCENT);
-        K('\\', BACKSLASH);
-        // }}}
-#undef K
-        default:
-            return key;
-    }
-}
-
 static inline void
 send_key_to_child(Window *w, int key, int mods, int action) {
     Screen *screen = w->render_data.screen;
@@ -152,14 +76,26 @@ send_key_to_child(Window *w, int key, int mods, int action) {
     }
 }
 
+static inline bool
+is_ascii_control_char(char c) {
+    return c == 0 || (1 <= c && c <= 31) || c == 127;
+}
+
 void
-on_key_input(int key, int scancode, int action, int mods) {
+on_key_input(int key, int scancode, int action, int mods, const char* text, int state UNUSED) {
     Window *w = active_window();
     if (!w) return;
+    if (global_state.in_sequence_mode) {
+        if (
+            action != GLFW_RELEASE &&
+            key != GLFW_KEY_LEFT_SHIFT && key != GLFW_KEY_RIGHT_SHIFT && key != GLFW_KEY_LEFT_ALT && key != GLFW_KEY_RIGHT_ALT && key != GLFW_KEY_LEFT_CONTROL && key != GLFW_KEY_RIGHT_CONTROL
+        ) call_boss(process_sequence, "iiii", key, scancode, action, mods);
+        return;
+    }
     Screen *screen = w->render_data.screen;
-    int lkey = get_localized_key(key, scancode);
+    bool has_text = text && !is_ascii_control_char(text[0]);
     if (action == GLFW_PRESS || action == GLFW_REPEAT) {
-        uint16_t qkey = key_map[lkey];
+        uint16_t qkey = (0 <= key && key < (ssize_t)arraysz(key_map)) ? key_map[key] : UINT8_MAX;
         bool special = false;
         if (qkey != UINT8_MAX) {
             qkey = SPECIAL_INDEX(qkey);
@@ -167,7 +103,7 @@ on_key_input(int key, int scancode, int action, int mods) {
         }
         /* printf("key: %s mods: %d special: %d\n", key_name(lkey), mods, special); */
         if (special) {
-            PyObject *ret = PyObject_CallMethod(global_state.boss, "dispatch_special_key", "iiii", lkey, scancode, action, mods);
+            PyObject *ret = PyObject_CallMethod(global_state.boss, "dispatch_special_key", "iiii", key, scancode, action, mods);
             if (ret == NULL) { PyErr_Print(); }
             else {
                 bool consumed = ret == Py_True;
@@ -176,20 +112,17 @@ on_key_input(int key, int scancode, int action, int mods) {
             }
         }
     }
+    if (action == GLFW_REPEAT && !screen->modes.mDECARM) return;
     if (screen->scrolled_by && action == GLFW_PRESS && !is_modifier_key(key)) {
         screen_history_scroll(screen, SCROLL_FULL, false);  // scroll back to bottom
     }
-#ifdef __APPLE__
-    if (!OPT(macos_option_as_alt) && IS_ALT_MODS(mods) && !(
-                key == GLFW_KEY_UP || key == GLFW_KEY_DOWN || key == GLFW_KEY_LEFT || key == GLFW_KEY_RIGHT)  // alt+arrow keys do not generate text on macOS
-       ) return;
-#endif
-    if (
-            action == GLFW_PRESS ||
-            (action == GLFW_REPEAT && screen->modes.mDECARM) ||
-            screen->modes.mEXTENDED_KEYBOARD
-       ) {
-        send_key_to_child(w, lkey, mods, action);
+    bool ok_to_send = action == GLFW_PRESS || action == GLFW_REPEAT || screen->modes.mEXTENDED_KEYBOARD;
+    if (ok_to_send) {
+        if (has_text) {
+            schedule_write_to_child(w->id, text, strlen(text));
+        } else {
+            send_key_to_child(w, key, mods, action);
+        }
     }
 }
 

kitty/keys.py

@@ -24,7 +24,9 @@ def modify_complex_key(name, amt):
     return modify_key_bytes(key_as_bytes(name), amt)
 
 
-control_codes = {}
+control_codes = {
+    defines.GLFW_KEY_BACKSPACE: b'\x08'
+}
 smkx_key_map = {}
 alt_codes = {
     defines.GLFW_KEY_TAB: b'\033\t',
@@ -37,6 +39,17 @@ shift_alt_codes[defines.GLFW_KEY_TAB] = key_as_bytes('kcbt')
 alt_mods = (defines.GLFW_MOD_ALT, defines.GLFW_MOD_SHIFT | defines.GLFW_MOD_ALT)
 ctrl_shift_mod = defines.GLFW_MOD_SHIFT | defines.GLFW_MOD_CONTROL
 ctrl_alt_mod = defines.GLFW_MOD_ALT | defines.GLFW_MOD_CONTROL
+SHIFTED_KEYS = {
+    defines.GLFW_KEY_TAB: key_as_bytes('kcbt'),
+    defines.GLFW_KEY_HOME: key_as_bytes('kHOM'),
+    defines.GLFW_KEY_END: key_as_bytes('kEND'),
+    defines.GLFW_KEY_LEFT: key_as_bytes('kLFT'),
+    defines.GLFW_KEY_RIGHT: key_as_bytes('kRIT'),
+    defines.GLFW_KEY_UP: key_as_bytes('kri'),
+    defines.GLFW_KEY_DOWN: key_as_bytes('kind'),
+}
+
+control_alt_codes = {}
 
 for kf, kn in {
     defines.GLFW_KEY_UP: 'kcuu1',
@@ -54,13 +67,20 @@ for kf, kn in {
     alt_codes[kf] = modify_complex_key(kn, 3)
     shift_alt_codes[kf] = modify_complex_key(kn, 4)
     control_codes[kf] = modify_complex_key(kn, 5)
+    control_alt_codes[kf] = modify_complex_key(kn, 7)
 for f in range(1, 13):
     kf = getattr(defines, 'GLFW_KEY_F{}'.format(f))
     kn = 'kf{}'.format(f)
     smkx_key_map[kf] = key_as_bytes(kn)
+    SHIFTED_KEYS[kf] = modify_complex_key(kn, 2)
     alt_codes[kf] = modify_complex_key(kn, 3)
     shift_alt_codes[kf] = modify_complex_key(kn, 4)
     control_codes[kf] = modify_complex_key(kn, 5)
+    control_alt_codes[kf] = modify_complex_key(kn, 7)
+for f in range(13, 26):
+    kf = getattr(defines, 'GLFW_KEY_F{}'.format(f))
+    kn = 'kf{}'.format(f)
+    smkx_key_map[kf] = key_as_bytes(kn)
 f = {k: k for k in '0123456789'}
 f.update({
     'COMMA': ',',
@@ -80,16 +100,6 @@ smkx_key_map[defines.GLFW_KEY_KP_ENTER] = b'\r'
 smkx_key_map[defines.GLFW_KEY_BACKSPACE] = key_as_bytes('kbs')
 smkx_key_map[defines.GLFW_KEY_TAB] = b'\t'
 
-SHIFTED_KEYS = {
-    defines.GLFW_KEY_TAB: key_as_bytes('kcbt'),
-    defines.GLFW_KEY_HOME: key_as_bytes('kHOM'),
-    defines.GLFW_KEY_END: key_as_bytes('kEND'),
-    defines.GLFW_KEY_LEFT: key_as_bytes('kLFT'),
-    defines.GLFW_KEY_RIGHT: key_as_bytes('kRIT'),
-    defines.GLFW_KEY_UP: key_as_bytes('kri'),
-    defines.GLFW_KEY_DOWN: key_as_bytes('kind'),
-}
-
 control_codes.update({
     k: (1 + i, )
     for i, k in
@@ -206,6 +216,7 @@ ASCII_C0_SHIFTED = {
 }
 CTRL_SHIFT_KEYS = {getattr(defines, 'GLFW_KEY_' + k): v for k, v in ASCII_C0_SHIFTED.items()}
 CTRL_ALT_KEYS = {getattr(defines, 'GLFW_KEY_' + k) for k in string.ascii_uppercase}
+all_control_alt_keys = set(CTRL_ALT_KEYS) | set(control_alt_codes)
 
 
 def key_to_bytes(key, smkx, extended, mods, action):
@@ -224,8 +235,11 @@ def key_to_bytes(key, smkx, extended, mods, action):
             m = UN_SHIFTED_PRINTABLE if mods == defines.GLFW_MOD_ALT else SHIFTED_PRINTABLE
             data.append(0o33)
             data.extend(m[key])
-    elif mods == ctrl_alt_mod and key in CTRL_ALT_KEYS:
-        data.append(0x1b), data.extend(control_codes[key])
+    elif mods == ctrl_alt_mod and key in all_control_alt_keys:
+        if key in CTRL_ALT_KEYS:
+            data.append(0x1b), data.extend(control_codes[key])
+        else:
+            data.extend(control_alt_codes[key])
     else:
         key_map = cursor_key_mode_map[smkx]
         x = key_map.get(key)
@@ -251,8 +265,12 @@ def get_shortcut(keymap, mods, key, scancode):
     return keymap.get((mods & 0b1111, key))
 
 
+def shortcut_matches(s, mods, key, scancode):
+    return s[0] & 0b1111 == mods & 0b1111 and s[1] == key
+
+
 def generate_key_table():
-    # To run this, use: python3 . -c "from kitty.keys import *; generate_key_table()"
+    # To run this, use: python3 . +runpy "from kitty.keys import *; generate_key_table()"
     import os
     from functools import partial
     f = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'keys.h'), 'w')

kitty/khrplatform.h

@@ -1,8 +1,7 @@
 #ifndef __khrplatform_h_
 #define __khrplatform_h_
-
 /*
-** Copyright (c) 2008-2009 The Khronos Group Inc.
+** Copyright (c) 2008-2018 The Khronos Group Inc.
 **
 ** Permission is hereby granted, free of charge, to any person obtaining a
 ** copy of this software and/or associated documentation files (the
@@ -23,21 +22,18 @@
 ** TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
 ** MATERIALS OR THE USE OR OTHER DEALINGS IN THE MATERIALS.
 */
-
 /* Khronos platform-specific types and definitions.
  *
- * $Revision: 32517 $ on $Date: 2016-03-11 02:41:19 -0800 (Fri, 11 Mar 2016) $
+ * The master copy of khrplatform.h is maintained in the Khronos EGL
+ * Registry repository at https://github.com/KhronosGroup/EGL-Registry
+ * The last semantic modification to khrplatform.h was at commit ID:
+ *      67a3e0864c2d75ea5287b9f3d2eb74a745936692
  *
  * Adopters may modify this file to suit their platform. Adopters are
  * encouraged to submit platform specific modifications to the Khronos
  * group so that they can be included in future versions of this file.
- * Please submit changes by sending them to the public Khronos Bugzilla
- * (http://khronos.org/bugzilla) by filing a bug against product
- * "Khronos (general)" component "Registry".
- *
- * A predefined template which fills in some of the bug fields can be
- * reached using http://tinyurl.com/khrplatform-h-bugreport, but you
- * must create a Bugzilla login first.
+ * Please submit changes by filing pull requests or issues on
+ * the EGL Registry repository linked above.
  *
  *
  * See the Implementer's Guidelines for information about where this file
@@ -91,7 +87,6 @@
  *                                  int arg1,
  *                                  int arg2) KHRONOS_APIATTRIBUTES;
  */
-
 /*-------------------------------------------------------------------------
  * Definition of KHRONOS_APICALL
  *-------------------------------------------------------------------------
@@ -106,7 +101,6 @@
 #else
 #   define KHRONOS_APICALL
 #endif
-
 /*-------------------------------------------------------------------------
  * Definition of KHRONOS_APIENTRY
  *-------------------------------------------------------------------------
@@ -119,7 +113,6 @@
 #else
 #   define KHRONOS_APIENTRY
 #endif
-
 /*-------------------------------------------------------------------------
  * Definition of KHRONOS_APIATTRIBUTES
  *-------------------------------------------------------------------------
@@ -130,13 +123,10 @@
 #else
 #define KHRONOS_APIATTRIBUTES
 #endif
-
 /*-------------------------------------------------------------------------
  * basic type definitions
  *-----------------------------------------------------------------------*/
 #if (defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L) || defined(__GNUC__) || defined(__SCO__) || defined(__USLC__)
-
-
 /*
  * Using <stdint.h>
  */
@@ -147,9 +137,7 @@ typedef int64_t                 khronos_int64_t;
 typedef uint64_t                khronos_uint64_t;
 #define KHRONOS_SUPPORT_INT64   1
 #define KHRONOS_SUPPORT_FLOAT   1
-
 #elif defined(__VMS ) || defined(__sgi)
-
 /*
  * Using <inttypes.h>
  */
@@ -160,9 +148,7 @@ typedef int64_t                 khronos_int64_t;
 typedef uint64_t                khronos_uint64_t;
 #define KHRONOS_SUPPORT_INT64   1
 #define KHRONOS_SUPPORT_FLOAT   1
-
 #elif defined(_WIN32) && !defined(__SCITECH_SNAP__)
-
 /*
  * Win32
  */
@@ -172,9 +158,7 @@ typedef __int64                 khronos_int64_t;
 typedef unsigned __int64        khronos_uint64_t;
 #define KHRONOS_SUPPORT_INT64   1
 #define KHRONOS_SUPPORT_FLOAT   1
-
 #elif defined(__sun__) || defined(__digital__)
-
 /*
  * Sun or Digital
  */
@@ -189,9 +173,7 @@ typedef unsigned long long int  khronos_uint64_t;
 #endif /* __arch64__ */
 #define KHRONOS_SUPPORT_INT64   1
 #define KHRONOS_SUPPORT_FLOAT   1
-
 #elif 0
-
 /*
  * Hypothetical platform with no float or int64 support
  */
@@ -199,9 +181,7 @@ typedef int                     khronos_int32_t;
 typedef unsigned int            khronos_uint32_t;
 #define KHRONOS_SUPPORT_INT64   0
 #define KHRONOS_SUPPORT_FLOAT   0
-
 #else
-
 /*
  * Generic fallback
  */
@@ -212,10 +192,7 @@ typedef int64_t                 khronos_int64_t;
 typedef uint64_t                khronos_uint64_t;
 #define KHRONOS_SUPPORT_INT64   1
 #define KHRONOS_SUPPORT_FLOAT   1
-
 #endif
-
-
 /*
  * Types that are (so far) the same on all platforms
  */
@@ -223,7 +200,6 @@ typedef signed   char          khronos_int8_t;
 typedef unsigned char          khronos_uint8_t;
 typedef signed   short int     khronos_int16_t;
 typedef unsigned short int     khronos_uint16_t;
-
 /*
  * Types that differ between LLP64 and LP64 architectures - in LLP64,
  * pointers are 64 bits, but 'long' is still 32 bits. Win64 appears
@@ -240,14 +216,12 @@ typedef unsigned long  int     khronos_uintptr_t;
 typedef signed   long  int     khronos_ssize_t;
 typedef unsigned long  int     khronos_usize_t;
 #endif
-
 #if KHRONOS_SUPPORT_FLOAT
 /*
  * Float type
  */
 typedef          float         khronos_float_t;
 #endif
-
 #if KHRONOS_SUPPORT_INT64
 /* Time types
  *
@@ -261,14 +235,12 @@ typedef          float         khronos_float_t;
 typedef khronos_uint64_t       khronos_utime_nanoseconds_t;
 typedef khronos_int64_t        khronos_stime_nanoseconds_t;
 #endif
-
 /*
  * Dummy value used to pad enum types to 32 bits.
  */
 #ifndef KHRONOS_MAX_ENUM
 #define KHRONOS_MAX_ENUM 0x7FFFFFFF
 #endif
-
 /*
  * Enumerated boolean type
  *
@@ -280,5 +252,4 @@ typedef enum {
     KHRONOS_TRUE  = 1,
     KHRONOS_BOOLEAN_ENUM_FORCE_SIZE = KHRONOS_MAX_ENUM
 } khronos_boolean_enum_t;
-
 #endif /* __khrplatform_h_ */

kitty/kitty.conf

@@ -1,5 +1,12 @@
 # vim:fileencoding=utf-8:ft=conf
 
+# You can include secondary config files via the "include" directive.
+# If you use a relative path for include, it is resolved with respect to the
+# location of the current config file. Note that environment variables
+# are expanded, so ${USER}.conf becomes name.conf if USER=name
+# For example:
+# include other.conf
+
 # Fonts {{{
 # Font family. You can also specify different fonts for the
 # bold/italic/bold-italic variants. By default they are derived automatically,
@@ -96,7 +103,7 @@ url_color #0087BD
 url_style curly
 
 # The modifier keys to press when clicking with the mouse on URLs to open the URL
-open_url_modifiers ctrl+shift
+open_url_modifiers kitty_mod
 
 # The program with which to open URLs that are clicked on. The special value "default" means to
 # use the operating system's default URL handler.
@@ -161,6 +168,14 @@ visual_bell_duration 0.0
 
 # Enable/disable the audio bell. Useful in environments that require silence.
 enable_audio_bell yes
+
+# Request window attention on bell.
+# Makes the dock icon bounce on macOS or the taskbar flash on linux.
+window_alert_on_bell yes
+
+# Show a bell symbol on the tab if a bell occurs in one of the windows in the
+# tab and the window is not the currently focused window
+bell_on_tab yes
 # }}}
 
 
@@ -177,6 +192,12 @@ initial_window_height  400
 # For a list of available layouts, see the README.
 enabled_layouts *
 
+# The step size (in units of cell width/cell height) to use when resizing
+# windows. The cells value is used for horizontal resizing and the lines value
+# for vertical resizing.
+window_resize_step_cells 2
+window_resize_step_lines 2
+
 # The width (in pts) of window borders. Will be rounded to the nearest number of pixels based on screen resolution.
 # Note that borders are displayed only when more than one window is visible. They are meant to separate multiple windows.
 window_border_width 1
@@ -193,6 +214,9 @@ active_border_color #00ff00
 # The color for the border of inactive windows
 inactive_border_color #cccccc
 
+# The color for the border of inactive windows in which a bell has occurred
+bell_border_color #ff5a00
+
 # Fade the text in inactive windows by the specified amount (a number between
 # zero and one, with 0 being fully faded).
 inactive_text_alpha 1.0
@@ -235,6 +259,10 @@ background       #000000
 # Be aware that using a value less than 1.0 is a (possibly significant) performance hit.
 background_opacity 1.0
 
+# How much to dim text that has the DIM/FAINT attribute set. 1.0 means no dimming and
+# 0.0 means fully dimmed (i.e. invisible).
+dim_opacity 0.75
+
 # The foreground for selections
 selection_foreground #000000
 
@@ -242,7 +270,8 @@ selection_foreground #000000
 selection_background #FFFACD
 
 # The 16 terminal colors. There are 8 basic colors, each color has a dull and
-# bright version.
+# bright version. You can also set the remaining colors from the 256 color table
+# as color16 to color256.
 
 # black
 color0   #000000
@@ -301,6 +330,14 @@ close_on_child_death no
 # Note that this even works over ssh connections.
 allow_remote_control no
 
+# Allow programs running in kitty to read and write from the clipboard. You can
+# control exactly which actions are allowed. The set of possible actions is:
+# write-clipboard read-clipboard write-primary read-primary
+# The default is to allow writing to the clipboard and primary selection. Note
+# that enabling the read functionality is a security risk as it means that any
+# program, even one running on a remote server via SSH can read your clipboard.
+clipboard_control write-clipboard write-primary
+
 # The value of the TERM environment variable to set. Changing this can break
 # many terminal programs, only change it if you know what you are doing, not
 # because you read some advice on Stack Overflow to change it.
@@ -313,6 +350,11 @@ term xterm-kitty
 # Keyboard shortcuts {{{
 # For a list of key names, see: http://www.glfw.org/docs/latest/group__keys.html
 # For a list of modifier names, see: http://www.glfw.org/docs/latest/group__mods.html
+
+# The value of kitty_mod is used as the modifier for all default shortcuts, you
+# can change it in your kitty.conf to change the modifiers for all the default
+# shortcuts.
+kitty_mod ctrl+shift
 #
 # You can use the special action no_op to unmap a keyboard shortcut that is
 # assigned in the default configuration.
@@ -321,63 +363,71 @@ term xterm-kitty
 # syntax below:
 # map key combine <separator> action1 <separator> action2 <separator> action3 ...
 # For example:
-# map ctrl+shift+e combine : new_window : next_layout
+# map kitty_mod+e combine : new_window : next_layout
 # this will create a new window and switch to the next available layout
+#
+# You can use multi-key shortcuts using the syntax shown below:
+# map key1>key2>key3 action
+# For example:
+# map ctrl+f>2 set_font_size 20
+# this will change the font size to 20 points when you press ctrl+f and then 2
 
 # Clipboard {{{
-map ctrl+shift+v        paste_from_clipboard
-map ctrl+shift+s        paste_from_selection
-map ctrl+shift+c        copy_to_clipboard
-map shift+insert        paste_from_selection
+map kitty_mod+v        paste_from_clipboard
+map kitty_mod+s        paste_from_selection
+map kitty_mod+c        copy_to_clipboard
+map shift+insert       paste_from_selection
 # You can also pass the contents of the current selection to any program using
 # pass_selection_to_program. By default, the system's open program is used, but
 # you can specify your own, for example:
-# map ctrl+shift+o      pass_selection_to_program firefox
-map ctrl+shift+o        pass_selection_to_program
+# map kitty_mod+o      pass_selection_to_program firefox
+map kitty_mod+o        pass_selection_to_program
 # }}}
 
 # Scrolling {{{
-map ctrl+shift+up        scroll_line_up
-map ctrl+shift+down      scroll_line_down
-map ctrl+shift+k         scroll_line_up
-map ctrl+shift+j         scroll_line_down
-map ctrl+shift+page_up   scroll_page_up
-map ctrl+shift+page_down scroll_page_down
-map ctrl+shift+home      scroll_home
-map ctrl+shift+end       scroll_end
-map ctrl+shift+h         show_scrollback
+map kitty_mod+up        scroll_line_up
+map kitty_mod+down      scroll_line_down
+map kitty_mod+k         scroll_line_up
+map kitty_mod+j         scroll_line_down
+map kitty_mod+page_up   scroll_page_up
+map kitty_mod+page_down scroll_page_down
+map kitty_mod+home      scroll_home
+map kitty_mod+end       scroll_end
+map kitty_mod+h         show_scrollback
 # }}}
 
 # Window management {{{
-map ctrl+shift+enter    new_window
-map ctrl+shift+n        new_os_window
-map ctrl+shift+w        close_window
-map ctrl+shift+]        next_window
-map ctrl+shift+[        previous_window
-map ctrl+shift+f        move_window_forward
-map ctrl+shift+b        move_window_backward
-map ctrl+shift+`        move_window_to_top
-map ctrl+shift+1        first_window
-map ctrl+shift+2        second_window
-map ctrl+shift+3        third_window
-map ctrl+shift+4        fourth_window
-map ctrl+shift+5        fifth_window
-map ctrl+shift+6        sixth_window
-map ctrl+shift+7        seventh_window
-map ctrl+shift+8        eighth_window
-map ctrl+shift+9        ninth_window
-map ctrl+shift+0        tenth_window
+map kitty_mod+enter    new_window
+map kitty_mod+n        new_os_window
+map kitty_mod+w        close_window
+map kitty_mod+]        next_window
+map kitty_mod+[        previous_window
+map kitty_mod+f        move_window_forward
+map kitty_mod+b        move_window_backward
+map kitty_mod+`        move_window_to_top
+map kitty_mod+r        start_resizing_window
+# Switching to a particular window
+map kitty_mod+1        first_window
+map kitty_mod+2        second_window
+map kitty_mod+3        third_window
+map kitty_mod+4        fourth_window
+map kitty_mod+5        fifth_window
+map kitty_mod+6        sixth_window
+map kitty_mod+7        seventh_window
+map kitty_mod+8        eighth_window
+map kitty_mod+9        ninth_window
+map kitty_mod+0        tenth_window
 # You can open a new window running an arbitrary program, for example:
-# map ctrl+shift+y      new_window mutt
+# map kitty_mod+y      new_window mutt
 #
 # You can pass the current selection to the new program by using the @selection placeholder
-# map ctrl+shift+y      new_window less @selection
+# map kitty_mod+y      new_window less @selection
 #
 # You can even send the contents of the current screen + history buffer as stdin using
 # the placeholders @text (which is the plain text) and @ansi (which includes text styling escape codes).
 # For only the current screen, use @screen or @ansi_screen.
 # For example, the following command opens the scrollback buffer in less in a new window.
-# map ctrl+shift+y      new_window @ansi less +G -R
+# map kitty_mod+y      new_window @ansi less +G -R
 #
 # You can open a new window with the current working directory set to the
 # working directory of the current window using
@@ -385,14 +435,14 @@ map ctrl+shift+0        tenth_window
 # }}}
 
 # Tab management {{{
-map ctrl+shift+right    next_tab
-map ctrl+shift+left     previous_tab
-map ctrl+shift+t        new_tab
-map ctrl+shift+q        close_tab
-map ctrl+shift+l        next_layout
-map ctrl+shift+.        move_tab_forward
-map ctrl+shift+,        move_tab_backward
-map ctrl+shift+alt+t    set_tab_title
+map kitty_mod+right    next_tab
+map kitty_mod+left     previous_tab
+map kitty_mod+t        new_tab
+map kitty_mod+q        close_tab
+map kitty_mod+l        next_layout
+map kitty_mod+.        move_tab_forward
+map kitty_mod+,        move_tab_backward
+map kitty_mod+alt+t    set_tab_title
 # You can also create shortcuts to go to specific tabs, with 1 being the first tab
 # map ctrl+alt+1          goto_tab 1
 # map ctrl+alt+2          goto_tab 2
@@ -408,23 +458,46 @@ map ctrl+shift+alt+t    set_tab_title
 # }}}
 
 # Font sizes {{{
-map ctrl+shift+equal    increase_font_size
-map ctrl+shift+minus    decrease_font_size
-map ctrl+shift+backspace restore_font_size
+map kitty_mod+equal    increase_font_size
+map kitty_mod+minus    decrease_font_size
+map kitty_mod+backspace restore_font_size
 # To setup shortcuts for specific font sizes, follow the example below:
-# map ctrl+shift+f6     set_font_size 10.0
-# map ctrl+shift+f7     set_font_size 20.5
+# map kitty_mod+f6     set_font_size 10.0
+# map kitty_mod+f7     set_font_size 20.5
+# }}}
+
+# Select and act on visible text {{{
+# Use the hints kitten to select text and either pass it to an external program or
+# insert it into the terminal or copy it to the clipboard.
+#
+# Open a currently visible URL using the keyboard. The program used to open the
+# URL is specified in open_url_with.
+map kitty_mod+e         kitten hints
+
+# Select a path/filename and insert it into the terminal. Useful, for instance to
+# run git commands on a filename output from a previous git command.
+map kitty_mod+p>f       kitten hints --type path --program -
+
+# Select a path/filename and open it with the default open program.
+map kitty_mod+p>shift+f kitten hints --type path
+
+# Select a line of text and insert it into the terminal. Use for the
+# output of things like: ls -1
+map kitty_mod+p>l       kitten hints --type line --program -
+
+# Select words and insert into terminal.
+map kitty_mod+p>w       kitten hints --type word --program -
+
+# The hints kitten has many more modes of operation that you can map to different
+# shortcuts. For a full description run: kitty +kitten hints --help
 # }}}
 
 # Miscellaneous {{{
-map ctrl+shift+f11      toggle_fullscreen
-map ctrl+shift+u        input_unicode_character
-map ctrl+shift+f2       edit_config_file
-# Open a currently visible URL using the keyboard. The program used to open the URL is specified in open_url_with.
-# You can customize how the URLs are detected and opened by specifying command line options to
-# url_hints. For example:
-# map ctrl+shift+e        run_simple_kitten text url_hints --program firefox --regex "http://[^ ]+"
-map ctrl+shift+e        run_simple_kitten text url_hints
+map kitty_mod+f11      toggle_fullscreen
+map kitty_mod+u        input_unicode_character
+map kitty_mod+f2       edit_config_file
+# Open the kitty shell in a new window/tab/overlay/os_window to control kitty using commands.
+map kitty_mod+escape   kitty_shell window
 
 # Sending arbitrary text on shortcut key presses
 # You can tell kitty to send arbitrary (UTF-8) encoded text to
@@ -444,6 +517,9 @@ map ctrl+shift+e        run_simple_kitten text url_hints
 # map ctrl+alt+a send_text application Word\x1bOH
 # }}}
 
+# You can have kitty remove all shortcut definition seen up to this point. Useful, for
+# instance, to remove the default shortcuts.
+clear_all_shortcuts no
 # }}}
 
 
@@ -451,8 +527,12 @@ map ctrl+shift+e        run_simple_kitten text url_hints
 
 # Change the color of the kitty window's titlebar on macOS. A value of "system"
 # means to use the default system color, a value of "background" means to use
-# the default background color and finally you can use an arbitrary color, such
-# as #12af59 or "red".
+# the background color of the currently active window and finally you can use
+# an arbitrary color, such as #12af59 or "red". WARNING: This option works by
+# using a hack, as there is no proper Cocoa API for it. It sets the background
+# color of the entire window and makes the titlebar transparent. As such it is
+# incompatible with background_opacity. If you want to use both, you are
+# probably better off just hiding the titlebar with macos_hide_titlebar.
 macos_titlebar_color system
 
 # Hide the kitty window's title bar on macOS.
@@ -463,4 +543,7 @@ macos_hide_titlebar no
 # break any Alt+key keyboard shortcuts in your terminal programs, but you
 # can use the macOS unicode input technique.
 macos_option_as_alt yes
+
+# Hide the kitty window from running tasks (alt-tab) on macOS.
+macos_hide_from_tasks no
 # }}}

kitty/layout.py

@@ -4,13 +4,14 @@
 
 from collections import namedtuple
 from functools import partial
-from itertools import islice
+from itertools import islice, repeat
 
 from .constants import WindowGeometry
 from .fast_data_types import (
-    Region, pt_to_px, set_active_window, swap_windows, viewport_for_window
+    Region, set_active_window, swap_windows, viewport_for_window
 )
 
+# Utils {{{
 central = Region((0, 0, 199, 199, 200, 200))
 cell_width = cell_height = 20
 
@@ -21,7 +22,7 @@ def idx_for_id(win_id, windows):
             return i
 
 
-def layout_dimension(start_at, length, cell_length, number_of_windows=1, border_length=0, margin_length=0, padding_length=0, left_align=False):
+def layout_dimension(start_at, length, cell_length, number_of_windows=1, border_length=0, margin_length=0, padding_length=0, left_align=False, bias=None):
     number_of_cells = length // cell_length
     border_length += padding_length
     space_needed_for_border = number_of_windows * 2 * border_length
@@ -37,8 +38,30 @@ def layout_dimension(start_at, length, cell_length, number_of_windows=1, border_
     if not left_align:
         pos += extra // 2
     pos += border_length + margin_length
-    inner_length = cells_per_window * cell_length
-    window_length = 2 * (border_length + margin_length) + inner_length
+
+    def calc_window_length(cells_in_window):
+        inner_length = cells_in_window * cell_length
+        return 2 * (border_length + margin_length) + inner_length
+
+    if bias is not None and number_of_windows > 1 and len(bias) == number_of_windows and cells_per_window > 5:
+        cells_map = [int(b * number_of_cells) for b in bias]
+        while min(cells_map) < 5:
+            maxi, mini = map(cells_map.index, (max(cells_map), min(cells_map)))
+            if maxi == mini:
+                break
+            cells_map[mini] += 1
+            cells_map[maxi] -= 1
+
+        extra = number_of_cells - sum(cells_map)
+        if extra:
+            cells_map[-1] += extra
+        for cells_per_window in cells_map:
+            window_length = calc_window_length(cells_per_window)
+            yield pos, cells_per_window
+            pos += window_length
+        return
+
+    window_length = calc_window_length(cells_per_window)
     extra = number_of_cells - (cells_per_window * number_of_windows)
     while number_of_windows > 0:
         number_of_windows -= 1
@@ -56,28 +79,140 @@ def process_overlaid_windows(all_windows):
     return overlaid_windows, windows
 
 
-class Layout:
+def window_geometry(xstart, xnum, ystart, ynum):
+    return WindowGeometry(left=xstart, top=ystart, xnum=xnum, ynum=ynum, right=xstart + cell_width * xnum, bottom=ystart + cell_height * ynum)
+
+
+def layout_single_window(margin_length, padding_length):
+    xstart, xnum = next(layout_dimension(central.left, central.width, cell_width, margin_length=margin_length, padding_length=padding_length))
+    ystart, ynum = next(layout_dimension(central.top, central.height, cell_height, margin_length=margin_length, padding_length=padding_length))
+    return window_geometry(xstart, xnum, ystart, ynum)
+
+
+def left_blank_rect(w, rects):
+    lt = w.geometry.left
+    if lt > central.left:
+        rects.append(Rect(central.left, central.top, lt, central.bottom + 1))
+
+
+def right_blank_rect(w, rects):
+    r = w.geometry.right
+    if r < central.right:
+        rects.append(Rect(r, central.top, central.right + 1, central.bottom + 1))
+
+
+def top_blank_rect(w, rects):
+    t = w.geometry.top
+    if t > central.top:
+        rects.append(Rect(central.left, central.top, central.right + 1, t))
+
+
+def bottom_blank_rect(w, rects):
+    b = w.geometry.bottom
+    # Need to use <= here as otherwise a single pixel row at the bottom of the
+    # window is sometimes not covered. See https://github.com/kovidgoyal/kitty/issues/506
+    if b <= central.bottom:
+        rects.append(Rect(central.left, b, central.right + 1, central.bottom + 1))
+
+
+def blank_rects_for_window(w):
+    ans = []
+    left_blank_rect(w, ans), top_blank_rect(w, ans), right_blank_rect(w, ans), bottom_blank_rect(w, ans)
+    return ans
+
+
+def safe_increment_bias(old_val, increment):
+    return max(0.1, min(old_val + increment, 0.9))
+
+
+def normalize_biases(biases):
+    s = sum(biases)
+    if s == 1:
+        return biases
+    return [x/s for x in biases]
+
+
+def distribute_indexed_bias(base_bias, index_bias_map):
+    if not index_bias_map:
+        return base_bias
+    ans = list(base_bias)
+    limit = len(ans)
+    for row, increment in index_bias_map.items():
+        if row >= limit or not increment:
+            continue
+        other_increment = -increment / (limit - 1)
+        ans = [safe_increment_bias(b, increment if i == row else other_increment) for i, b in enumerate(ans)]
+    return normalize_biases(ans)
+
+
+def variable_bias(num_windows, candidate):
+    return distribute_indexed_bias(list(repeat(1/(num_windows), num_windows)), candidate)
+
+
+# }}}
+
+
+class Layout:  # {{{
 
     name = None
     needs_window_borders = True
     only_active_window_visible = False
 
-    def __init__(self, os_window_id, tab_id, opts, border_width):
+    def __init__(self, os_window_id, tab_id, margin_width, padding_width, border_width, layout_opts=''):
         self.os_window_id = os_window_id
         self.tab_id = tab_id
         self.set_active_window_in_os_window = partial(set_active_window, os_window_id, tab_id)
         self.swap_windows_in_os_window = partial(swap_windows, os_window_id, tab_id)
-        self.opts = opts
         self.border_width = border_width
-        self.margin_width = pt_to_px(opts.window_margin_width)
-        self.padding_width = pt_to_px(opts.window_padding_width)
+        self.margin_width = margin_width
+        self.padding_width = padding_width
         # A set of rectangles corresponding to the blank spaces at the edges of
         # this layout, i.e. spaces that are not covered by any window
-        self.blank_rects = ()
+        self.blank_rects = []
+        self.layout_opts = self.parse_layout_opts(layout_opts)
+        self.full_name = self.name + ((':' + layout_opts) if layout_opts else '')
+        self.remove_all_biases()
 
-    def nth_window(self, all_windows, num):
+    def bias_increment_for_cell(self, is_horizontal):
+        self._set_dimensions()
+        if is_horizontal:
+            return (cell_width + 1) / central.width
+        return (cell_height + 1) / central.height
+
+    def apply_bias(self, idx, increment_as_percent, num_windows, is_horizontal):
+        return False
+
+    def remove_all_biases(self):
+        return False
+
+    def modify_size_of_window(self, all_windows, window_id, increment, is_horizontal=True):
+        idx = idx_for_id(window_id, all_windows)
+        if idx is None:
+            return False
+        w = all_windows[idx]
+        windows = process_overlaid_windows(all_windows)[1]
+        idx = idx_for_id(w.id, windows)
+        if idx is None:
+            idx = idx_for_id(w.overlay_window_id, windows)
+        if idx is not None:
+            return self.apply_bias(idx, increment, len(windows), is_horizontal)
+        return False
+
+    def parse_layout_opts(self, layout_opts):
+        if not layout_opts:
+            return {}
+        ans = {}
+        for x in layout_opts.split(';'):
+            k, v = x.partition('=')[::2]
+            if k and v:
+                ans[k] = v
+        return ans
+
+    def nth_window(self, all_windows, num, make_active=True):
         windows = process_overlaid_windows(all_windows)[1]
         w = windows[min(num, len(windows) - 1)]
+        if not make_active:
+            return w
         active_window_idx = idx_for_id(w.id, all_windows)
         return self.set_active_window(all_windows, active_window_idx)
 
@@ -178,10 +313,12 @@ class Layout:
         self.set_active_window_in_os_window(active_window_idx)
         return active_window_idx
 
-    def __call__(self, all_windows, active_window_idx):
+    def _set_dimensions(self):
         global central, cell_width, cell_height
         central, tab_bar, vw, vh, cell_width, cell_height = viewport_for_window(self.os_window_id)
 
+    def __call__(self, all_windows, active_window_idx):
+        self._set_dimensions()
         active_window = all_windows[active_window_idx]
         overlaid_windows, windows = process_overlaid_windows(all_windows)
         if overlaid_windows:
@@ -193,19 +330,25 @@ class Layout:
         else:
             windows = all_windows
         self.update_visibility(all_windows, active_window, overlaid_windows)
+        self.blank_rects = []
         self.do_layout(windows, active_window_idx)
         return idx_for_id(active_window.id, all_windows)
 
     # Utils {{{
-    def xlayout(self, num):
+    def layout_single_window(self, w):
+        wg = layout_single_window(self.margin_width, self.padding_width)
+        w.set_geometry(0, wg)
+        self.blank_rects = blank_rects_for_window(w)
+
+    def xlayout(self, num, bias=None):
         return layout_dimension(
             central.left, central.width, cell_width, num, self.border_width,
-            margin_length=self.margin_width, padding_length=self.padding_width)
+            margin_length=self.margin_width, padding_length=self.padding_width, bias=bias)
 
-    def ylayout(self, num, left_align=True):
+    def ylayout(self, num, left_align=True, bias=None):
         return layout_dimension(
             central.top, central.height, cell_height, num, self.border_width, left_align=left_align,
-            margin_length=self.margin_width, padding_length=self.padding_width)
+            margin_length=self.margin_width, padding_length=self.padding_width, bias=bias)
 
     def simple_blank_rects(self, first_window, last_window):
         br = self.blank_rects
@@ -220,80 +363,81 @@ class Layout:
 
     def do_layout(self, windows, active_window_idx):
         raise NotImplementedError()
+# }}}
 
 
-def window_geometry(xstart, xnum, ystart, ynum):
-    return WindowGeometry(left=xstart, top=ystart, xnum=xnum, ynum=ynum, right=xstart + cell_width * xnum, bottom=ystart + cell_height * ynum)
-
-
-def layout_single_window(margin_length, padding_length):
-    xstart, xnum = next(layout_dimension(central.left, central.width, cell_width, margin_length=margin_length, padding_length=padding_length))
-    ystart, ynum = next(layout_dimension(central.top, central.height, cell_height, margin_length=margin_length, padding_length=padding_length))
-    return window_geometry(xstart, xnum, ystart, ynum)
-
-
-def left_blank_rect(w, rects):
-    lt = w.geometry.left
-    if lt > central.left:
-        rects.append(Rect(central.left, central.top, lt, central.bottom + 1))
-
-
-def right_blank_rect(w, rects):
-    r = w.geometry.right
-    if r < central.right:
-        rects.append(Rect(r, central.top, central.right + 1, central.bottom + 1))
-
-
-def top_blank_rect(w, rects):
-    t = w.geometry.top
-    if t > central.top:
-        rects.append(Rect(central.left, central.top, central.right + 1, t))
-
-
-def bottom_blank_rect(w, rects):
-    b = w.geometry.bottom
-    if b < central.bottom:
-        rects.append(Rect(central.left, b, central.right + 1, central.bottom + 1))
-
-
-def blank_rects_for_window(w):
-    ans = []
-    left_blank_rect(w, ans), top_blank_rect(w, ans), right_blank_rect(w, ans), bottom_blank_rect(w, ans)
-    return ans
-
-
-class Stack(Layout):
+class Stack(Layout):  # {{{
 
     name = 'stack'
     needs_window_borders = False
     only_active_window_visible = True
 
     def do_layout(self, windows, active_window_idx):
-        self.blank_rects = []
         wg = layout_single_window(self.margin_width, self.padding_width)
         for i, w in enumerate(windows):
             w.set_geometry(i, wg)
             if w.is_visible_in_layout:
                 self.blank_rects = blank_rects_for_window(w)
+# }}}
 
 
-class Tall(Layout):
+class Tall(Layout):  # {{{
 
     name = 'tall'
+    vlayout = Layout.ylayout
+    main_is_horizontal = True
+
+    def remove_all_biases(self):
+        self.main_bias = list(self.layout_opts['bias'])
+        self.biased_map = {}
+        return True
+
+    def variable_layout(self, num_windows, biased_map):
+        num_windows -= 1
+        return self.vlayout(num_windows, bias=variable_bias(num_windows, biased_map) if num_windows > 1 else None)
+
+    def apply_bias(self, idx, increment, num_windows, is_horizontal):
+        if self.main_is_horizontal == is_horizontal:
+            before = self.main_bias
+            if idx == 0:
+                self.main_bias = [safe_increment_bias(self.main_bias[0], increment), safe_increment_bias(self.main_bias[1], -increment)]
+            else:
+                self.main_bias = [safe_increment_bias(self.main_bias[0], -increment), safe_increment_bias(self.main_bias[1], increment)]
+            self.main_bias = normalize_biases(self.main_bias)
+            after = self.main_bias
+        else:
+            if idx == 0 or num_windows < 3:
+                return False
+            idx -= 1
+            before_layout = list(self.variable_layout(num_windows, self.biased_map))
+            candidate = self.biased_map.copy()
+            before = candidate.get(idx, 0)
+            candidate[idx] = after = before + increment
+            if before_layout == list(self.variable_layout(num_windows, candidate)):
+                return False
+            self.biased_map = candidate
+
+        return before != after
+
+    def parse_layout_opts(self, layout_opts):
+        ans = Layout.parse_layout_opts(self, layout_opts)
+        try:
+            ans['bias'] = int(ans.get('bias', 50)) / 100
+        except Exception:
+            ans['bias'] = 0.5
+        ans['bias'] = max(0.1, min(ans['bias'], 0.9))
+        ans['bias'] = ans['bias'], 1.0 - ans['bias']
+        return ans
 
     def do_layout(self, windows, active_window_idx):
-        self.blank_rects = []
         if len(windows) == 1:
-            wg = layout_single_window(self.margin_width, self.padding_width)
-            windows[0].set_geometry(0, wg)
-            self.blank_rects = blank_rects_for_window(windows[0])
-            return
-        xlayout = self.xlayout(2)
+            return self.layout_single_window(windows[0])
+        xlayout = self.xlayout(2, bias=self.main_bias)
         xstart, xnum = next(xlayout)
-        ystart, ynum = next(self.ylayout(1))
+        ystart, ynum = next(self.vlayout(1))
         windows[0].set_geometry(0, window_geometry(xstart, xnum, ystart, ynum))
         xstart, xnum = next(xlayout)
-        ylayout = self.ylayout(len(windows) - 1)
+        ylayout = self.variable_layout(len(windows), self.biased_map)
         for i, (w, (ystart, ynum)) in enumerate(zip(islice(windows, 1, None), ylayout)):
             w.set_geometry(i + 1, window_geometry(xstart, xnum, ystart, ynum))
             # right bottom blank rect
@@ -305,19 +449,98 @@ class Tall(Layout):
         self.between_blank_rect(windows[0], windows[1])
         # left bottom blank rect
         self.bottom_blank_rect(windows[0])
+# }}}
 
 
-class Grid(Tall):
+class Fat(Tall):  # {{{
+
+    name = 'fat'
+    vlayout = Layout.xlayout
+    main_is_horizontal = False
+
+    def do_layout(self, windows, active_window_idx):
+        if len(windows) == 1:
+            return self.layout_single_window(windows[0])
+        xstart, xnum = next(self.xlayout(1))
+        ylayout = self.ylayout(2, bias=self.main_bias)
+        ystart, ynum = next(ylayout)
+        windows[0].set_geometry(0, window_geometry(xstart, xnum, ystart, ynum))
+        xlayout = self.variable_layout(len(windows), self.biased_map)
+        ystart, ynum = next(ylayout)
+        for i, (w, (xstart, xnum)) in enumerate(zip(islice(windows, 1, None), xlayout)):
+            w.set_geometry(i + 1, window_geometry(xstart, xnum, ystart, ynum))
+            if i > 0:
+                # bottom between blank rect
+                self.between_blank_rect(windows[i - 1], windows[i])
+
+        # left, top and right blank rects
+        self.simple_blank_rects(windows[0], windows[-1])
+        # top bottom blank rect
+        self.bottom_blank_rect(windows[0])
+        # bottom blank rect
+        self.blank_rects.append(Rect(windows[0].geometry.left, windows[0].geometry.bottom, windows[-1].geometry.right, central.bottom + 1))
+# }}}
+
+
+class Grid(Layout):  # {{{
 
     name = 'grid'
 
-    def do_layout(self, windows, active_window_idx):
-        n = len(windows)
-        if n < 4:
-            return Tall.do_layout(self, windows, active_window_idx)
-        self.blank_rects = []
+    def remove_all_biases(self):
+        self.biased_rows = {}
+        self.biased_cols = {}
+        return True
+
+    def variable_layout(self, layout_func, num_windows, biased_map):
+        return layout_func(num_windows, bias=variable_bias(num_windows, biased_map) if num_windows > 1 else None)
+
+    def apply_bias(self, idx, increment, num_windows, is_horizontal):
+        b = self.biased_cols if is_horizontal else self.biased_rows
+        ncols, nrows, special_rows, special_col = self.calc_grid_size(num_windows)
+
+        def position_for_window_idx(idx):
+            row_num = col_num = 0
+
+            def on_col_done(col_windows):
+                nonlocal col_num, row_num
+                row_num = 0
+                col_num += 1
+
+            for window_idx, xstart, xnum, ystart, ynum in self.layout_windows(
+                    num_windows, nrows, ncols, special_rows, special_col, on_col_done):
+                if idx == window_idx:
+                    return row_num, col_num
+                row_num += 1
+
+        row_num, col_num = position_for_window_idx(idx)
+
+        if is_horizontal:
+            b = self.biased_cols
+            if ncols < 2:
+                return False
+            bias_idx = col_num
+            layout_func = self.xlayout
+            attr = 'biased_cols'
+        else:
+            b = self.biased_rows
+            if max(nrows, special_rows) < 2:
+                return False
+            bias_idx = row_num
+            layout_func = self.ylayout
+            attr = 'biased_rows'
+
+        before_layout = list(self.variable_layout(layout_func, num_windows, b))
+        candidate = b.copy()
+        before = candidate.get(bias_idx, 0)
+        candidate[bias_idx] = before + increment
+        if before_layout == list(self.variable_layout(layout_func, num_windows, candidate)):
+            return False
+        setattr(self, attr, candidate)
+        return True
+
+    def calc_grid_size(self, n):
         if n <= 5:
-            ncols = 2
+            ncols = 1 if n == 1 else 2
         else:
             for ncols in range(3, (n // 2) + 1):
                 if ncols * ncols >= n:
@@ -325,27 +548,44 @@ class Grid(Tall):
         nrows = n // ncols
         special_rows = n - (nrows * (ncols - 1))
         special_col = 0 if special_rows < nrows else ncols - 1
+        return ncols, nrows, special_rows, special_col
 
+    def layout_windows(self, num_windows, nrows, ncols, special_rows, special_col, on_col_done=lambda col_windows: None):
         # Distribute windows top-to-bottom, left-to-right (i.e. in columns)
-        xlayout = self.xlayout(ncols)
-        yvals_normal = tuple(self.ylayout(nrows))
-        yvals_special = yvals_normal if special_rows == nrows else tuple(self.ylayout(special_rows))
-
-        winmap = list(enumerate(windows))
+        xlayout = self.variable_layout(self.xlayout, ncols, self.biased_cols)
+        yvals_normal = tuple(self.variable_layout(self.ylayout, nrows, self.biased_rows))
+        yvals_special = yvals_normal if special_rows == nrows else tuple(self.variable_layout(self.ylayout, special_rows, self.biased_rows))
         pos = 0
-        win_col_map = []
         for col in range(ncols):
             rows = special_rows if col == special_col else nrows
             yl = yvals_special if col == special_col else yvals_normal
             xstart, xnum = next(xlayout)
             col_windows = []
-            for (ystart, ynum), (window_idx, w) in zip(yl, winmap[pos:pos + rows]):
-                w.set_geometry(window_idx, window_geometry(xstart, xnum, ystart, ynum))
-                col_windows.append(w)
-            # bottom blank rect
-            self.bottom_blank_rect(w)
+            for i, (ystart, ynum) in enumerate(yl):
+                window_idx = pos + i
+                yield window_idx, xstart, xnum, ystart, ynum
+                col_windows.append(window_idx)
             pos += rows
+            on_col_done(col_windows)
+
+    def do_layout(self, windows, active_window_idx):
+        n = len(windows)
+        if n == 1:
+            return self.layout_single_window(windows[0])
+        ncols, nrows, special_rows, special_col = self.calc_grid_size(n)
+
+        win_col_map = []
+
+        def on_col_done(col_windows):
+            col_windows = [windows[i] for i in col_windows]
             win_col_map.append(col_windows)
+            # bottom blank rect
+            self.bottom_blank_rect(col_windows[-1])
+
+        for window_idx, xstart, xnum, ystart, ynum in self.layout_windows(
+                len(windows), nrows, ncols, special_rows, special_col, on_col_done):
+            w = windows[window_idx]
+            w.set_geometry(window_idx, window_geometry(xstart, xnum, ystart, ynum))
 
         # left, top and right blank rects
         self.simple_blank_rects(windows[0], windows[-1])
@@ -358,23 +598,40 @@ class Grid(Tall):
 class Vertical(Layout):
 
     name = 'vertical'
+    main_is_horizontal = False
+    vlayout = Layout.ylayout
+
+    def variable_layout(self, num_windows, biased_map):
+        return self.vlayout(num_windows, bias=variable_bias(num_windows, biased_map) if num_windows else None)
+
+    def remove_all_biases(self):
+        self.biased_map = {}
+        return True
+
+    def apply_bias(self, idx, increment, num_windows, is_horizontal):
+        if self.main_is_horizontal != is_horizontal:
+            return False
+        if num_windows < 2:
+            return False
+        before_layout = list(self.variable_layout(num_windows, self.biased_map))
+        candidate = self.biased_map.copy()
+        before = candidate.get(idx, 0)
+        candidate[idx] = before + increment
+        if before_layout == list(self.variable_layout(num_windows, candidate)):
+            return False
+        self.biased_map = candidate
+        return True
 
     def do_layout(self, windows, active_window_idx):
-        self.blank_rects = []
         window_count = len(windows)
         if window_count == 1:
-            wg = layout_single_window(self.margin_width, self.padding_width)
-            windows[0].set_geometry(0, wg)
-            self.blank_rects = blank_rects_for_window(windows[0])
-            return
+            return self.layout_single_window(windows[0])
 
         xlayout = self.xlayout(1)
         xstart, xnum = next(xlayout)
-        ylayout = self.ylayout(window_count)
-
-        for i in range(window_count):
-            ystart, ynum = next(ylayout)
-            windows[i].set_geometry(i, window_geometry(xstart, xnum, ystart, ynum))
+        ylayout = self.variable_layout(window_count, self.biased_map)
+        for i, (w, (ystart, ynum)) in enumerate(zip(windows, ylayout)):
+            w.set_geometry(i, window_geometry(xstart, xnum, ystart, ynum))
             # bottom blank rect
             self.bottom_blank_rect(windows[i])
 
@@ -382,26 +639,22 @@ class Vertical(Layout):
         self.simple_blank_rects(windows[0], windows[-1])
 
 
-class Horizontal(Layout):
+class Horizontal(Vertical):
 
     name = 'horizontal'
+    main_is_horizontal = True
+    vlayout = Layout.xlayout
 
     def do_layout(self, windows, active_window_idx):
-        self.blank_rects = []
         window_count = len(windows)
         if window_count == 1:
-            wg = layout_single_window(self.margin_width, self.padding_width)
-            windows[0].set_geometry(0, wg)
-            self.blank_rects = blank_rects_for_window(windows[0])
-            return
+            return self.layout_single_window(windows[0])
 
-        xlayout = self.xlayout(window_count)
+        xlayout = self.variable_layout(window_count, self.biased_map)
         ylayout = self.ylayout(1)
         ystart, ynum = next(ylayout)
-
-        for i in range(window_count):
-            xstart, xnum = next(xlayout)
-            windows[i].set_geometry(i, window_geometry(xstart, xnum, ystart, ynum))
+        for i, (w, (xstart, xnum)) in enumerate(zip(windows, xlayout)):
+            w.set_geometry(i, window_geometry(xstart, xnum, ystart, ynum))
             if i > 0:
                 # between blank rect
                 self.between_blank_rect(windows[i - 1], windows[i])
@@ -410,6 +663,29 @@ class Horizontal(Layout):
         self.simple_blank_rects(windows[0], windows[-1])
         # bottom blank rect
         self.blank_rects.append(Rect(windows[0].geometry.left, windows[0].geometry.bottom, windows[-1].geometry.right, central.bottom + 1))
+# }}}
 
 
+# Instantiation {{{
+
 all_layouts = {o.name: o for o in globals().values() if isinstance(o, type) and issubclass(o, Layout) and o is not Layout}
+
+
+def create_layout_object_for(name, os_window_id, tab_id, margin_width, padding_width, border_width, layout_opts=''):
+    key = name, os_window_id, tab_id, margin_width, padding_width, border_width, layout_opts
+    ans = create_layout_object_for.cache.get(key)
+    if ans is None:
+        name, layout_opts = name.partition(':')[::2]
+        ans = create_layout_object_for.cache[key] = all_layouts[name](os_window_id, tab_id, margin_width, padding_width, border_width, layout_opts)
+    return ans
+
+
+create_layout_object_for.cache = {}
+
+
+def evict_cached_layouts(tab_id):
+    remove = [key for key in create_layout_object_for.cache if key[2] == tab_id]
+    for key in remove:
+        del create_layout_object_for.cache[key]
+
+# }}}

kitty/line-buf.c

@@ -149,7 +149,7 @@ set_attribute(LineBuf *self, PyObject *args) {
 #define set_attribute_doc "set_attribute(which, val) -> Set the attribute on all cells in the line."
     unsigned int shift, val;
     if (!PyArg_ParseTuple(args, "II", &shift, &val)) return NULL;
-    if (shift < DECORATION_SHIFT || shift > STRIKE_SHIFT) { PyErr_SetString(PyExc_ValueError, "Unknown attribute"); return NULL; }
+    if (shift < DECORATION_SHIFT || shift > DIM_SHIFT) { PyErr_SetString(PyExc_ValueError, "Unknown attribute"); return NULL; }
     linebuf_set_attribute(self, shift, val);
     Py_RETURN_NONE;
 }
@@ -414,7 +414,7 @@ get_line(LineBuf *self, index_type y) {
 
 static PyObject*
 as_text(LineBuf *self, PyObject *args) {
-    as_text_generic(args, self, get_line, self->ynum, self->xnum, callback, as_ansi);
+    as_text_generic(args, self, get_line, self->ynum, self->xnum);
 }
 
 

kitty/line.c

@@ -515,7 +515,7 @@ set_attribute(Line *self, PyObject *args) {
 #define set_attribute_doc "set_attribute(which, val) -> Set the attribute on all cells in the line."
     unsigned int shift, val;
     if (!PyArg_ParseTuple(args, "II", &shift, &val)) return NULL;
-    if (shift < DECORATION_SHIFT || shift > STRIKE_SHIFT) { PyErr_SetString(PyExc_ValueError, "Unknown attribute"); return NULL; }
+    if (shift < DECORATION_SHIFT || shift > DIM_SHIFT) { PyErr_SetString(PyExc_ValueError, "Unknown attribute"); return NULL; }
     set_attribute_on_line(self->cells, shift, val, self->xnum);
     Py_RETURN_NONE;
 }

kitty/lineops.h

@@ -72,7 +72,6 @@ void linebuf_mark_line_dirty(LineBuf *self, index_type y);
 void linebuf_mark_line_clean(LineBuf *self, index_type y);
 unsigned int linebuf_char_width_at(LineBuf *self, index_type x, index_type y);
 void linebuf_refresh_sprite_positions(LineBuf *self);
-bool historybuf_resize(HistoryBuf *self, index_type lines);
 void historybuf_add_line(HistoryBuf *self, const Line *line);
 void historybuf_rewrap(HistoryBuf *self, HistoryBuf *other);
 void historybuf_init_line(HistoryBuf *self, index_type num, Line *l);
@@ -82,14 +81,15 @@ void historybuf_refresh_sprite_positions(HistoryBuf *self);
 void historybuf_clear(HistoryBuf *self);
 
 
-#define as_text_generic(args, container, get_line, lines, columns, callback, as_ansi) { \
+#define as_text_generic(args, container, get_line, lines, columns) { \
     PyObject *callback; \
-    int as_ansi = 0; \
-    if (!PyArg_ParseTuple(args, "O|p", &callback, &as_ansi)) return NULL; \
+    int as_ansi = 0, insert_wrap_markers = 0; \
+    if (!PyArg_ParseTuple(args, "O|pp", &callback, &as_ansi, &insert_wrap_markers)) return NULL; \
     PyObject *ret = NULL, *t = NULL; \
     Py_UCS4 *buf = NULL; \
     PyObject *nl = PyUnicode_FromString("\n"); \
-    if (nl == NULL) goto end; \
+    PyObject *cr = PyUnicode_FromString("\r"); \
+    if (nl == NULL || cr == NULL) goto end; \
     if (as_ansi) { \
         buf = malloc(sizeof(Py_UCS4) * columns * 100); \
         if (buf == NULL) { PyErr_NoMemory(); goto end; } \
@@ -110,9 +110,14 @@ void historybuf_clear(HistoryBuf *self);
         if (t == NULL) goto end; \
         ret = PyObject_CallFunctionObjArgs(callback, t, NULL); \
         Py_DECREF(t); if (ret == NULL) goto end; Py_DECREF(ret); \
+        if (insert_wrap_markers) { \
+            ret = PyObject_CallFunctionObjArgs(callback, cr, NULL); \
+            if (ret == NULL) goto end; \
+            Py_CLEAR(ret); \
+        }\
     } \
 end: \
-    Py_CLEAR(nl); free(buf); \
+    Py_CLEAR(nl); Py_CLEAR(cr); free(buf); \
     if (PyErr_Occurred()) return NULL; \
     Py_RETURN_NONE; \
 }

kitty/main.py

@@ -39,12 +39,10 @@ def init_graphics():
 def run_app(opts, args):
     set_scale(opts.box_drawing_scale)
     set_options(opts, is_wayland, args.debug_gl, args.debug_font_fallback)
-    if is_macos:
-        from .fast_data_types import macos_change_titlebar_color
-        macos_change_titlebar_color(opts.macos_titlebar_color)
-    with cached_values_for('main') as cached_values:
-        w, h = initial_window_size(opts, cached_values)
+    with cached_values_for(run_app.cached_values_name) as cached_values:
+        w, h = run_app.initial_window_size(opts, cached_values)
         window_id = create_os_window(w, h, appname, args.name or args.cls or appname, args.cls or appname, load_all_shaders)
+        run_app.first_window_callback(opts, window_id)
         startup_ctx = init_startup_notification(window_id)
         show_window(window_id)
         if not is_wayland and not is_macos:  # no window icons on wayland
@@ -59,6 +57,11 @@ def run_app(opts, args):
             boss.destroy()
 
 
+run_app.cached_values_name = 'main'
+run_app.first_window_callback = lambda opts, window_id: None
+run_app.initial_window_size = initial_window_size
+
+
 def ensure_osx_locale():
     # Ensure the LANG env var is set. See
     # https://github.com/kovidgoyal/kitty/issues/90
@@ -154,9 +157,6 @@ def _main():
         return
     if getattr(args, 'detach', False):
         detach()
-    if args.cmd:
-        exec(args.cmd)
-        return
     if args.replay_commands:
         from kitty.client import main
         main(args.replay_commands)
@@ -165,7 +165,9 @@ def _main():
         is_first = single_instance(args.instance_group)
         if not is_first:
             import json
-            data = {'cmd': 'new_instance', 'args': tuple(sys.argv), 'startup_id': os.environ.get('DESKTOP_STARTUP_ID')}
+            data = {'cmd': 'new_instance', 'args': tuple(sys.argv),
+                    'startup_id': os.environ.get('DESKTOP_STARTUP_ID'),
+                    'cwd': os.getcwd()}
             data = json.dumps(data, ensure_ascii=False).encode('utf-8')
             single_instance.socket.sendall(data)
             return

kitty/mouse.c

@@ -93,24 +93,51 @@ encode_mouse_event(Window *w, int button, MouseAction action, int mods) {
 
 // }}}
 
+static inline double
+window_left(Window *w) {
+    return w->geometry.left - OPT(window_padding_width) * (global_state.logical_dpi_x / 72.0);
+}
+
+static inline double
+window_right(Window *w) {
+    return w->geometry.right + OPT(window_padding_width) * (global_state.logical_dpi_x / 72.0);
+}
+
+static inline double
+window_top(Window *w) {
+    return w->geometry.top - OPT(window_padding_width) * (global_state.logical_dpi_y / 72.0);
+}
+
+static inline double
+window_bottom(Window *w) {
+    return w->geometry.bottom + OPT(window_padding_width) * (global_state.logical_dpi_y / 72.0);
+}
+
 static inline bool
 contains_mouse(Window *w) {
-    WindowGeometry *g = &w->geometry;
     double x = global_state.callback_os_window->mouse_x, y = global_state.callback_os_window->mouse_y;
-    return (w->visible && g->left <= x && x <= g->right && g->top <= y && y <= g->bottom);
+    return (w->visible && window_left(w) <= x && x <= window_right(w) && window_top(w) <= y && y <= window_bottom(w));
 }
 
 static inline bool
 cell_for_pos(Window *w, unsigned int *x, unsigned int *y) {
     WindowGeometry *g = &w->geometry;
-    unsigned int qx = (unsigned int)((double)(global_state.callback_os_window->mouse_x - g->left) / global_state.cell_width);
-    unsigned int qy = (unsigned int)((double)(global_state.callback_os_window->mouse_y - g->top) / global_state.cell_height);
-    bool ret = false;
     Screen *screen = w->render_data.screen;
-    if (screen && qx <= screen->columns && qy <= screen->lines) {
-        *x = qx; *y = qy; ret = true;
+    if (!screen) return false;
+    unsigned int qx = 0, qy = 0;
+    double mouse_x = global_state.callback_os_window->mouse_x;
+    double mouse_y = global_state.callback_os_window->mouse_y;
+    double left = window_left(w), top = window_top(w), right = window_right(w), bottom = window_bottom(w);
+    if (mouse_x < left || mouse_y < top || mouse_x > right || mouse_y > bottom) return false;
+    if (mouse_x >= g->right) qx = screen->columns - 1;
+    else if (mouse_x >= g->left) qx = (unsigned int)((double)(mouse_x - g->left) / global_state.cell_width);
+    if (mouse_y >= g->bottom) qy = screen->lines - 1;
+    else if (mouse_y >= g->top) qy = (unsigned int)((double)(mouse_y - g->top) / global_state.cell_height);
+    if (qx < screen->columns && qy < screen->lines) {
+        *x = qx; *y = qy;
+        return true;
     }
-    return ret;
+    return false;
 }
 
 #define HANDLER(name) static inline void name(Window UNUSED *w, int UNUSED button, int UNUSED modifiers, unsigned int UNUSED window_idx)
@@ -120,7 +147,7 @@ update_drag(bool from_button, Window *w, bool is_release, int modifiers) {
     Screen *screen = w->render_data.screen;
     if (from_button) {
         if (is_release) screen_update_selection(screen, w->mouse_cell_x, w->mouse_cell_y, true);
-        else screen_start_selection(screen, w->mouse_cell_x, w->mouse_cell_y, modifiers == (int)OPT(rectangle_select_modifiers), EXTEND_CELL);
+        else screen_start_selection(screen, w->mouse_cell_x, w->mouse_cell_y, modifiers == (int)OPT(rectangle_select_modifiers) || modifiers == ((int)OPT(rectangle_select_modifiers) | GLFW_MOD_SHIFT), EXTEND_CELL);
     } else if (screen->selection.in_progress) {
         screen_update_selection(screen, w->mouse_cell_x, w->mouse_cell_y, false);
     }
@@ -130,9 +157,10 @@ update_drag(bool from_button, Window *w, bool is_release, int modifiers) {
 bool
 drag_scroll(Window *w, OSWindow *frame) {
     unsigned int margin = global_state.cell_height / 2;
+    double left = window_left(w), top = window_top(w), right = window_right(w), bottom = window_bottom(w);
     double x = frame->mouse_x, y = frame->mouse_y;
-    if (y < w->geometry.top || y > w->geometry.bottom) return false;
-    if (x < w->geometry.left || x > w->geometry.right) return false;
+    if (y < top || y > bottom) return false;
+    if (x < left || x > right) return false;
     bool upwards = y <= (w->geometry.top + margin);
     if (upwards || y >= w->geometry.bottom - margin) {
         Screen *screen = w->render_data.screen;
@@ -232,9 +260,10 @@ multi_click(Window *w, unsigned int count) {
     index_type start, end;
     bool found_selection = false;
     SelectionExtendMode mode = EXTEND_CELL;
+    unsigned int y1 = w->mouse_cell_y, y2 = w->mouse_cell_y;
     switch(count) {
         case 2:
-            found_selection = screen_selection_range_for_word(screen, w->mouse_cell_x, w->mouse_cell_y, &start, &end);
+            found_selection = screen_selection_range_for_word(screen, w->mouse_cell_x, &y1, &y2, &start, &end);
             mode = EXTEND_WORD;
             break;
         case 3:
@@ -245,8 +274,8 @@ multi_click(Window *w, unsigned int count) {
             break;
     }
     if (found_selection) {
-        screen_start_selection(screen, start, w->mouse_cell_y, false, mode);
-        screen_update_selection(screen, end, w->mouse_cell_y, false);
+        screen_start_selection(screen, start, y1, false, mode);
+        screen_update_selection(screen, end, y2, false);
     }
 }
 
@@ -282,7 +311,7 @@ HANDLER(handle_button_event) {
     Screen *screen = w->render_data.screen;
     if (!screen) return;
     bool handle_in_kitty = (
-            modifiers == GLFW_MOD_SHIFT ||
+            modifiers == GLFW_MOD_SHIFT || modifiers == ((int)OPT(rectangle_select_modifiers) | GLFW_MOD_SHIFT) ||
             screen->modes.mouse_tracking_mode == 0 ||
             button == GLFW_MOUSE_BUTTON_MIDDLE ||
             (modifiers == (int)OPT(open_url_modifiers) && button == GLFW_MOUSE_BUTTON_LEFT)

kitty/parser.c

@@ -341,6 +341,9 @@ dispatch_osc(Screen *screen, PyObject DUMP_UNUSED *dump_callback) {
             case 119:
                 SET_COLOR(set_dynamic_color);
                 break;
+            case 52:
+                DISPATCH_OSC(clipboard_control);
+                break;
             default:
                 REPORT_ERROR("Unknown OSC code: %u", code);
                 break;
@@ -766,17 +769,28 @@ dispatch_dcs(Screen *screen, PyObject DUMP_UNUSED *dump_callback) {
             }
             break;
         case '@':
-            if (startswith(screen->parser_buf + 1, screen->parser_buf_pos - 2, "kitty-cmd{")) {
+#define CMD_PREFIX "kitty-cmd{"
+            if (startswith(screen->parser_buf + 1, screen->parser_buf_pos - 2, CMD_PREFIX)) {
                 PyObject *cmd = PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, screen->parser_buf + 10, screen->parser_buf_pos - 10);
                 if (cmd != NULL) {
                     REPORT_OSC2(screen_handle_cmd, (char)screen->parser_buf[0], cmd);
                     screen_handle_cmd(screen, cmd);
                     Py_DECREF(cmd);
                 } else PyErr_Clear();
+#undef CMD_PREFIX
+#define PRINT_PREFIX "kitty-print|"
+            } else if (startswith(screen->parser_buf + 1, screen->parser_buf_pos - 1, PRINT_PREFIX)) {
+                PyObject *msg = PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, screen->parser_buf + sizeof(PRINT_PREFIX), screen->parser_buf_pos - sizeof(PRINT_PREFIX));
+                if (msg != NULL) {
+                    REPORT_OSC2(screen_handle_print, (char)screen->parser_buf[0], msg);
+                    screen_handle_print(screen, msg);
+                    Py_DECREF(msg);
+                } else PyErr_Clear();
             } else {
                 REPORT_ERROR("Unrecognized DCS @ code: 0x%x", screen->parser_buf[1]);
             }
             break;
+#undef PRINT_PREFIX
         default:
             REPORT_ERROR("Unrecognized DCS code: 0x%x", screen->parser_buf[0]);
             break;
@@ -816,7 +830,7 @@ parse_graphics_code(Screen *screen, PyObject UNUSED *dump_callback) {
     enum KEYS key = 'a';
     static GraphicsCommand g;
     unsigned int i, code;
-    unsigned long lcode;
+    uint64_t lcode;
     bool is_negative;
     memset(&g, 0, sizeof(g));
     static uint8_t payload[4096];

kitty/remote_control.py

@@ -4,470 +4,14 @@
 
 import json
 import re
-import socket
 import sys
 import types
 from functools import partial
 
 from .cli import emph, parse_args
-from .config import parse_send_text_bytes
 from .constants import appname, version
-from .tabs import SpecialWindow
-from .utils import non_blocking_read, parse_address_spec, read_with_timeout
-
-
-class MatchError(ValueError):
-
-    hide_traceback = True
-
-    def __init__(self, expression, target='windows'):
-        ValueError.__init__(self, 'No matching {} for expression: {}'.format(target, expression))
-
-
-def cmd(short_desc, desc=None, options_spec=None, no_response=False, argspec='...'):
-
-    def w(func):
-        func.short_desc = short_desc
-        func.argspec = argspec
-        func.desc = desc or short_desc
-        func.name = func.__name__[4:].replace('_', '-')
-        func.options_spec = options_spec
-        func.is_cmd = True
-        func.impl = lambda: globals()[func.__name__[4:]]
-        func.no_response = no_response
-        return func
-    return w
-
-
-def parse_subcommand_cli(func, args):
-    opts, items = parse_args(args[1:], (func.options_spec or '\n').format, func.argspec, func.desc, '{} @ {}'.format(appname, func.name))
-    return opts, items
-
-
-MATCH_WINDOW_OPTION = '''\
---match -m
-The window to match. Match specifications are of the form:
-|_ field:regexp|. Where field can be one of: id, title, pid, cwd, cmdline.
-You can use the |_ ls| command to get a list of windows. Note that for
-numeric fields such as id and pid the expression is interpreted as a number,
-not a regular expression.
-'''
-MATCH_TAB_OPTION = '''\
---match -m
-The tab to match. Match specifications are of the form:
-|_ field:regexp|. Where field can be one of: id, title, pid, cwd, cmdline.
-You can use the |_ ls| command to get a list of tabs. Note that for
-numeric fields such as id and pid the expression is interpreted as a number,
-not a regular expression. When using title or id, first a matching tab is
-looked for and if not found a matching window is looked for, and the tab
-for that window is used.
-'''
-
-
-# ls {{{
-@cmd(
-    'List all tabs/windows',
-    'List all windows. The list is returned as JSON tree. The top-level is a list of'
-    ' operating system {appname} windows. Each OS window has an |_ id| and a list'
-    ' of |_ tabs|. Each tab has its own |_ id|, a |_ title| and a list of |_ windows|.'
-    ' Each window has an |_ id|, |_ title|, |_ current working directory|, |_ process id (PID)| and'
-    ' |_ command-line| of the process running in the window.\n\n'
-    'You can use these criteria to select windows/tabs for the other commands.'.format(appname=appname),
-    argspec=''
-)
-def cmd_ls(global_opts, opts, args):
-    pass
-
-
-def ls(boss, window):
-    data = list(boss.list_os_windows())
-    data = json.dumps(data, indent=2, sort_keys=True)
-    return data
-# }}}
-
-
-# set_font_size {{{
-@cmd(
-    'Set the font size in all windows',
-    'Sets the font size to the specified size, in pts.',
-    argspec='FONT_SIZE'
-)
-def cmd_set_font_size(global_opts, opts, args):
-    try:
-        return {'size': float(args[0])}
-    except IndexError:
-        raise SystemExit('No font size specified')
-
-
-def set_font_size(boss, window, payload):
-    boss.set_font_size(payload['size'])
-# }}}
-
-
-# send_text {{{
-@cmd(
-    'Send arbitrary text to specified windows',
-    'Send arbitrary text to specified windows. The text follows Python'
-    ' escaping rules. So you can use escapes like |_ \\x1b| to send control codes'
-    ' and |_ \\u21fa| to send unicode characters. If you use the |_ --match| option'
-    ' the text will be sent to all matched windows. By default, text is sent to'
-    ' only the currently active window.',
-    options_spec=MATCH_WINDOW_OPTION + '''\n
---stdin
-type=bool-set
-Read the text to be sent from |_ stdin|. Note that in this case the text is sent as is,
-not interpreted for escapes. If stdin is a terminal, you can press Ctrl-D to end reading.
-
-
---from-file
-Path to a file whose contents you wish to send. Note that in this case the file contents
-are sent as is, not interpreted for escapes.
-''',
-    no_response=True,
-    argspec='[TEXT TO SEND]'
-)
-def cmd_send_text(global_opts, opts, args):
-    limit = 1024
-    ret = {'match': opts.match, 'is_binary': False}
-
-    def pipe(src=sys.stdin):
-        ret['is_binary'] = True
-        import select
-        with non_blocking_read() as fd:
-            keep_going = True
-            while keep_going:
-                rd = select.select([fd], [], [])
-                if rd:
-                    data = sys.stdin.buffer.read()
-                    if not data:
-                        break
-                    data = data.decode('utf-8')
-                    if '\x04' in data:
-                        data = data[:data.index('\x04')]
-                        keep_going = False
-                    while data:
-                        ret['text'] = data[:limit]
-                        yield ret
-                        data = data[limit:]
-                else:
-                    break
-
-    def chunks(text):
-        ret['is_binary'] = False
-        while text:
-            ret['text'] = text[:limit]
-            yield ret
-            text = text[limit:]
-
-    def file_pipe(path):
-        ret['is_binary'] = True
-        with open(path, encoding='utf-8') as f:
-            while True:
-                data = f.read(limit)
-                if not data:
-                    break
-                ret['text'] = data
-                yield ret
-
-    sources = []
-    if opts.stdin:
-        sources.append(pipe())
-
-    if opts.from_file:
-        sources.append(file_pipe(opts.from_file))
-
-    text = ' '.join(args)
-    sources.append(chunks(text))
-
-    def chain():
-        for src in sources:
-            yield from src
-    return chain()
-
-
-def send_text(boss, window, payload):
-    windows = [boss.active_window]
-    match = payload['match']
-    if match:
-        windows = tuple(boss.match_windows(match))
-    data = payload['text'].encode('utf-8') if payload['is_binary'] else parse_send_text_bytes(payload['text'])
-    for window in windows:
-        if window is not None:
-            window.write_to_child(data)
-# }}}
-
-
-# set_window_title {{{
-@cmd(
-    'Set the window title',
-    'Set the title for the specified window(s). If you use the |_ --match| option'
-    ' the title will be set for all matched windows. By default, only the window'
-    ' in which the command is run is affected. If you do not specify a title, the'
-    ' last title set by the child process running in the window will be used.',
-    options_spec=MATCH_WINDOW_OPTION,
-    argspec='TITLE ...'
-)
-def cmd_set_window_title(global_opts, opts, args):
-    return {'title': ' '.join(args), 'match': opts.match}
-
-
-def set_window_title(boss, window, payload):
-    windows = [window or boss.active_window]
-    match = payload['match']
-    if match:
-        windows = tuple(boss.match_windows(match))
-        if not windows:
-            raise MatchError(match)
-    for window in windows:
-        if window:
-            window.set_title(payload['title'])
-# }}}
-
-
-# set_tab_title {{{
-@cmd(
-    'Set the tab title',
-    'Set the title for the specified tab(s). If you use the |_ --match| option'
-    ' the title will be set for all matched tabs. By default, only the tab'
-    ' in which the command is run is affected. If you do not specify a title, the'
-    ' title of the currently active window in the tab is used.',
-    options_spec=MATCH_TAB_OPTION,
-    argspec='TITLE ...'
-)
-def cmd_set_tab_title(global_opts, opts, args):
-    return {'title': ' '.join(args), 'match': opts.match}
-
-
-def set_tab_title(boss, window, payload):
-    match = payload['match']
-    if match:
-        tabs = tuple(boss.match_tabs(match))
-        if not tabs:
-            raise MatchError(match, 'tabs')
-    else:
-        tabs = [boss.tab_for_window(window) if window else boss.active_tab]
-    for tab in tabs:
-        if tab:
-            tab.set_title(payload['title'])
-# }}}
-
-
-# close_window {{{
-@cmd(
-    'Close the specified window(s)',
-    options_spec=MATCH_WINDOW_OPTION + '''\n
---self
-type=bool-set
-If specified close the window this command is run in, rather than the active window.
-''',
-    argspec=''
-)
-def cmd_close_window(global_opts, opts, args):
-    return {'match': opts.match, 'self': opts.self}
-
-
-def close_window(boss, window, payload):
-    match = payload['match']
-    if match:
-        windows = tuple(boss.match_windows(match))
-        if not windows:
-            raise MatchError(match)
-    else:
-        windows = [window if window and payload['self'] else boss.active_window]
-    for window in windows:
-        if window:
-            boss.close_window(window)
-# }}}
-
-
-# close_tab {{{
-@cmd(
-    'Close the specified tab(s)',
-    options_spec=MATCH_TAB_OPTION + '''\n
---self
-type=bool-set
-If specified close the tab this command is run in, rather than the active tab.
-''',
-    argspec=''
-)
-def cmd_close_tab(global_opts, opts, args):
-    return {'match': opts.match, 'self': opts.self}
-
-
-def close_tab(boss, window, payload):
-    match = payload['match']
-    if match:
-        tabs = tuple(boss.match_tabs(match))
-        if not tabs:
-            raise MatchError(match, 'tabs')
-    else:
-        tabs = [boss.tab_for_window(window) if window and payload['self'] else boss.active_tab]
-    for tab in tabs:
-        if window:
-            if tab:
-                boss.close_tab(tab)
-# }}}
-
-
-# new_window {{{
-@cmd(
-    'Open new window',
-    'Open a new window in the specified tab. If you use the |_ --match| option'
-    ' the first matching tab is used. Otherwise the currently active tab is used.'
-    ' Prints out the id of the newly opened window. Any command line arguments'
-    ' are assumed to be the command line used to run in the new window, if none'
-    ' are provided, the default shell is run. For example:\n'
-    '|_ kitty @ new-window --title Email mutt|',
-    options_spec=MATCH_TAB_OPTION + '''\n
---title
-The title for the new window. By default it will use the title set by the
-program running in it.
-
-
---cwd
-The initial working directory for the new window.
-
-
---keep-focus
-type=bool-set
-Keep the current window focused instead of switching to the newly opened window
-
-
---new-tab
-type=bool-set
-Open a new tab
-
-
---tab-title
-When using --new-tab set the title of the tab.
-''',
-    argspec='[CMD ...]'
-)
-def cmd_new_window(global_opts, opts, args):
-    return {'match': opts.match, 'title': opts.title, 'cwd': opts.cwd,
-            'new_tab': opts.new_tab, 'tab_title': opts.tab_title,
-            'keep_focus': opts.keep_focus, 'args': args or []}
-
-
-def new_window(boss, window, payload):
-    w = SpecialWindow(cmd=payload['args'] or None, override_title=payload['title'], cwd=payload['cwd'])
-    old_window = boss.active_window
-    if payload['new_tab']:
-        boss._new_tab(w)
-        tab = boss.active_tab
-        if payload['tab_title']:
-            tab.set_title(payload['tab_title'])
-        wid = boss.active_window.id
-        if payload['keep_focus'] and old_window:
-            boss.set_active_window(old_window)
-        return str(wid)
-
-    match = payload['match']
-    if match:
-        tabs = tuple(boss.match_tabs(match))
-        if not tabs:
-            raise MatchError(match, 'tabs')
-    else:
-        tabs = [boss.active_tab]
-    tab = tabs[0]
-    w = tab.new_special_window(w)
-    if payload['keep_focus'] and old_window:
-        boss.set_active_window(old_window)
-    return str(w.id)
-# }}}
-
-
-# focus_window {{{
-@cmd(
-    'Focus the specified window',
-    options_spec=MATCH_WINDOW_OPTION,
-    argspec='',
-)
-def cmd_focus_window(global_opts, opts, args):
-    return {'match': opts.match}
-
-
-def focus_window(boss, window, payload):
-    windows = [window or boss.active_window]
-    match = payload['match']
-    if match:
-        windows = tuple(boss.match_windows(match))
-        if not windows:
-            raise MatchError(match)
-    for window in windows:
-        if window:
-            boss.set_active_window(window)
-            break
-# }}}
-
-
-# focus_tab {{{
-@cmd(
-    'Focus the specified tab',
-    'The active window in the specified tab will be focused.',
-    options_spec=MATCH_TAB_OPTION,
-    argspec='',
-)
-def cmd_focus_tab(global_opts, opts, args):
-    return {'match': opts.match}
-
-
-def focus_tab(boss, window, payload):
-    match = payload['match']
-    tabs = tuple(boss.match_tabs(match))
-    if not tabs:
-        raise MatchError(match, 'tabs')
-    tab = tabs[0]
-    boss.set_active_tab(tab)
-# }}}
-
-
-# get_text {{{
-@cmd(
-    'Get text from the specified window',
-    options_spec=MATCH_WINDOW_OPTION + '''\n
---extent
-default=screen
-choices=screen, all, selection
-What text to get. The default of screen means all text currently on the screen. all means
-all the screen+scrollback and selection means currently selected text.
-
-
---ansi
-type=bool-set
-By default, only plain text is returned. If you specify this flag, the text will
-include the formatting escape codes for colors/bold/italic/etc. Note that when
-getting the current selection, the result is always plain text.
-
-
---self
-type=bool-set
-If specified get text from the window this command is run in, rather than the active window.
-''',
-    argspec=''
-)
-def cmd_get_text(global_opts, opts, args):
-    return {'match': opts.match, 'extent': opts.extent, 'ansi': opts.ansi, 'self': opts.self}
-
-
-def get_text(boss, window, payload):
-    match = payload['match']
-    if match:
-        windows = tuple(boss.match_windows(match))
-        if not windows:
-            raise MatchError(match)
-    else:
-        windows = [window if window and payload['self'] else boss.active_window]
-    window = windows[0]
-    if payload['extent'] == 'selection':
-        ans = window.text_for_selection()
-    else:
-        ans = window.as_text(as_ansi=bool(payload['ansi']), add_history=True)
-    return ans
-# }}}
-
-
-cmap = {v.name: v for v in globals().values() if hasattr(v, 'is_cmd')}
+from .utils import parse_address_spec, read_with_timeout
+from .cmds import cmap, parse_subcommand_cli
 
 
 def handle_cmd(boss, window, cmd):
@@ -495,9 +39,14 @@ will only work if this process is run within an existing kitty window.
 '''.format, appname=appname)
 
 
-def do_io(to, send, no_response):
+def encode_send(send):
     send = ('@kitty-cmd' + json.dumps(send)).encode('ascii')
-    send = b'\x1bP' + send + b'\x1b\\'
+    return b'\x1bP' + send + b'\x1b\\'
+
+
+def do_io(to, send, no_response):
+    import socket
+    send = encode_send(send)
     if to:
         family, address = parse_address_spec(to)[:2]
         s = socket.socket(family)
@@ -534,8 +83,10 @@ def do_io(to, send, no_response):
     return response
 
 
-def main(args):
-    all_commands = tuple(sorted(cmap))
+all_commands = tuple(sorted(cmap))
+
+
+def parse_rc_args(args):
     cmds = ('  |G {}|\n    {}'.format(cmap[c].name, cmap[c].short_desc) for c in all_commands)
     msg = (
         'Control {appname} by sending it commands. Add'
@@ -545,10 +96,16 @@ def main(args):
         '{appname} @ |_ command| -h'
     ).format(appname=appname, cmds='\n'.join(cmds))
 
-    global_opts, items = parse_args(args[1:], global_options_spec, 'command ...', msg, '{} @'.format(appname))
+    return parse_args(args[1:], global_options_spec, 'command ...', msg, '{} @'.format(appname))
+
+
+def main(args):
+    global_opts, items = parse_rc_args(args)
 
     if not items:
-        raise SystemExit('You must specify a command')
+        from kitty.shell import main
+        main(global_opts)
+        return
     cmd = items[0]
     try:
         func = cmap[cmd]
@@ -573,5 +130,8 @@ def main(args):
         if response.get('tb'):
             print(response['tb'], file=sys.stderr)
         raise SystemExit(response['error'])
-    if 'data' in response:
-        print(response['data'])
+    data = response.get('data')
+    if data is not None:
+        if func.string_return_is_error and isinstance(data, str):
+            raise SystemExit(data)
+        print(data)