Merge branch 'develop' into gsouquet-scroll-to-live-reset-hash

docs/room-list-store.md

@@ -6,7 +6,7 @@ It's so complicated it needs its own README.
 
 Legend:
 * Orange = External event.
-* Purple = Deterministic flow. 
+* Purple = Deterministic flow.
 * Green = Algorithm definition.
 * Red = Exit condition/point.
 * Blue = Process definition.
@@ -24,8 +24,8 @@ algorithm to call, instead of having all the logic in the room list store itself
 
 
 Tag sorting is effectively the comparator supplied to the list algorithm. This gives the list algorithm
-the power to decide when and how to apply the tag sorting, if at all. For example, the importance algorithm, 
+the power to decide when and how to apply the tag sorting, if at all. For example, the importance algorithm,
-later described in this document, heavily uses the list ordering behaviour to break the tag into categories. 
+later described in this document, heavily uses the list ordering behaviour to break the tag into categories.
 Each category then gets sorted by the appropriate tag sorting algorithm.
 
 ### Tag sorting algorithm: Alphabetical
@@ -36,7 +36,7 @@ useful.
 
 ### Tag sorting algorithm: Manual
 
-Manual sorting makes use of the `order` property present on all tags for a room, per the 
+Manual sorting makes use of the `order` property present on all tags for a room, per the
 [Matrix specification](https://matrix.org/docs/spec/client_server/r0.6.0#room-tagging). Smaller values
 of `order` cause rooms to appear closer to the top of the list.
 
@@ -74,7 +74,7 @@ relative (perceived) importance to the user:
   set to 'All Messages'.
 * **Bold**: The room has unread messages waiting for the user. Essentially this is a grey room without
   a badge/notification count (or 'Mentions Only'/'Muted').
-* **Idle**: No useful (see definition of useful above) activity has occurred in the room since the user 
+* **Idle**: No useful (see definition of useful above) activity has occurred in the room since the user
   last read it.
 
 Conveniently, each tag gets ordered by those categories as presented: red rooms appear above grey, grey
@@ -82,7 +82,7 @@ above bold, etc.
 
 Once the algorithm has determined which rooms belong in which categories, the tag sorting algorithm
 gets applied to each category in a sub-list fashion. This should result in the red rooms (for example)
-being sorted alphabetically amongst each other as well as the grey rooms sorted amongst each other, but 
+being sorted alphabetically amongst each other as well as the grey rooms sorted amongst each other, but
 collectively the tag will be sorted into categories with red being at the top.
 
 ## Sticky rooms
@@ -103,48 +103,62 @@ receive another notification which causes the room to move into the topmost posi
 above the sticky room will move underneath to allow for the new room to take the top slot, maintaining
 the sticky room's position.
 
-Though only applicable to the importance algorithm, the sticky room is not aware of category boundaries 
+Though only applicable to the importance algorithm, the sticky room is not aware of category boundaries
-and thus the user can see a shift in what kinds of rooms move around their selection. An example would 
+and thus the user can see a shift in what kinds of rooms move around their selection. An example would
-be the user having 4 red rooms, the user selecting the third room (leaving 2 above it), and then having 
+be the user having 4 red rooms, the user selecting the third room (leaving 2 above it), and then having
-the rooms above it read on another device. This would result in 1 red room and 1 other kind of room 
+the rooms above it read on another device. This would result in 1 red room and 1 other kind of room
 above the sticky room as it will try to maintain 2 rooms above the sticky room.
 
 An exception for the sticky room placement is when there's suddenly not enough rooms to maintain the placement
 exactly. This typically happens if the user selects a room and leaves enough rooms where it cannot maintain
 the N required rooms above the sticky room. In this case, the sticky room will simply decrease N as needed.
-The N value will never increase while selection remains unchanged: adding a bunch of rooms after having 
+The N value will never increase while selection remains unchanged: adding a bunch of rooms after having
 put the sticky room in a position where it's had to decrease N will not increase N.
 
 ## Responsibilities of the store
 
-The store is responsible for the ordering, upkeep, and tracking of all rooms. The room list component simply gets 
+The store is responsible for the ordering, upkeep, and tracking of all rooms. The room list component simply gets
-an object containing the tags it needs to worry about and the rooms within. The room list component will 
+an object containing the tags it needs to worry about and the rooms within. The room list component will
-decide which tags need rendering (as it commonly filters out empty tags in most cases), and will deal with 
+decide which tags need rendering (as it commonly filters out empty tags in most cases), and will deal with
 all kinds of filtering.
 
 ## Filtering
 
-Filters are provided to the store as condition classes, which are then passed along to the algorithm
+Filters are provided to the store as condition classes and have two major kinds: Prefilters and Runtime.
-implementations. The implementations then get to decide how to actually filter the rooms, however in
-practice the base `Algorithm` class deals with the filtering in a more optimized/generic way.
 
-The results of filters get cached to avoid needlessly iterating over potentially thousands of rooms,
+Prefilters flush out rooms which shouldn't appear to the algorithm implementations. Typically this is
-as the old room list store does. When a filter condition changes, it emits an update which (in this
+due to some higher order room list filtering (such as spaces or tags) deliberately exposing a subset of
-case) the `Algorithm` class will pick up and act accordingly. Typically, this also means filtering a 
+rooms to the user. The algorithm implementations will not see a room being prefiltered out.
+
+Runtime filters are used for more dynamic filtering, such as the user filtering by room name. These
+filters are passed along to the algorithm implementations where those implementations decide how and
+when to apply the filter. In practice, the base `Algorithm` class ends up doing the heavy lifting for
+optimization reasons.
+
+The results of runtime filters get cached to avoid needlessly iterating over potentially thousands of
+rooms, as the old room list store does. When a filter condition changes, it emits an update which (in this
+case) the `Algorithm` class will pick up and act accordingly. Typically, this also means filtering a
 minor subset where possible to avoid over-iterating rooms.
 
 All filter conditions are considered "stable" by the consumers, meaning that the consumer does not
 expect a change in the condition unless the condition says it has changed. This is intentional to
 maintain the caching behaviour described above.
 
+One might ask why we don't just use prefilter conditions for everything, and the answer is one of slight
+subtlety: in the cases of prefilters we are knowingly exposing the user to a workspace-style UX where
+room notifications are self-contained within that workspace. Runtime filters tend to not want to affect
+visible notification counts (as it doesn't want the room header to suddenly be confusing to the user as
+they type), and occasionally UX like "found 2/12 rooms" is desirable. If prefiltering were used instead,
+the notification counts would vary while the user was typing and "found 2/12" UX would not be possible.
+
 ## Class breakdowns
 
-The `RoomListStore` is the major coordinator of various algorithm implementations, which take care 
+The `RoomListStore` is the major coordinator of various algorithm implementations, which take care
-of the various `ListAlgorithm` and `SortingAlgorithm` options. The `Algorithm` class is responsible 
+of the various `ListAlgorithm` and `SortingAlgorithm` options. The `Algorithm` class is responsible
-for figuring out which tags get which rooms, as Matrix specifies them as a reverse map: tags get 
+for figuring out which tags get which rooms, as Matrix specifies them as a reverse map: tags get
-defined on rooms and are not defined as a collection of rooms (unlike how they are presented to the 
+defined on rooms and are not defined as a collection of rooms (unlike how they are presented to the
-user). Various list-specific utilities are also included, though they are expected to move somewhere 
+user). Various list-specific utilities are also included, though they are expected to move somewhere
-more general when needed. For example, the `membership` utilities could easily be moved elsewhere 
+more general when needed. For example, the `membership` utilities could easily be moved elsewhere
 as needed.
 
 The various bits throughout the room list store should also have jsdoc of some kind to help describe

res/css/views/rooms/_RoomPreviewBar.scss

@@ -40,6 +40,35 @@ limitations under the License.
         word-break: break-word;
     }
 
+    .mx_RoomPreviewBar_reason {
+        text-align: left;
+        background-color: $primary-bg-color;
+        border: 1px solid $invite-reason-border-color;
+        border-radius: 10px;
+        padding: 0 16px 12px 16px;
+        margin: 5px 0 20px 0;
+
+        div {
+            pointer-events: none;
+        }
+
+        .mx_EventTile_msgOption {
+            display: none;
+        }
+
+        .mx_MatrixChat_useCompactLayout & {
+            padding-top: 9px;
+        }
+
+        &.mx_EventTilePreview_faded {
+            cursor: pointer;
+
+            .mx_SenderProfile, .mx_EventTile_avatar {
+                opacity: 0.3;
+            }
+        }
+    }
+
     .mx_Spinner {
         width: auto;
         height: auto;

res/css/views/voip/_CallView.scss

@@ -55,7 +55,7 @@ limitations under the License.
         }
     }
 
-    .mx_CallView_voice_holdText {
+    .mx_CallView_holdTransferContent {
         padding-top: 10px;
         padding-bottom: 25px;
     }
@@ -82,7 +82,7 @@ limitations under the License.
     }
 }
 
-.mx_CallView_voice_hold {
+.mx_CallView_voice .mx_CallView_holdTransferContent {
     // This masks the avatar image so when it's blurred, the edge is still crisp
     .mx_CallView_voice_avatarContainer {
         border-radius: 2000px;
@@ -91,7 +91,7 @@ limitations under the License.
     }
 }
 
-.mx_CallView_voice_holdText {
+.mx_CallView_holdTransferContent {
     height: 20px;
     padding-top: 20px;
     padding-bottom: 15px;
@@ -142,7 +142,7 @@ limitations under the License.
     }
 }
 
-.mx_CallView_video_holdContent {
+.mx_CallView_video .mx_CallView_holdTransferContent {
     position: absolute;
     top: 50%;
     left: 50%;

res/themes/dark/css/_dark.scss

@@ -209,6 +209,8 @@ $message-body-panel-fg-color: $primary-fg-color;
 // Appearance tab colors
 $appearance-tab-border-color: $room-highlight-color;
 
+$invite-reason-border-color: $room-highlight-color;
+
 // blur amounts for left left panel (only for element theme, used in _mods.scss)
 $roomlist-background-blur-amount: 60px;
 $groupFilterPanel-background-blur-amount: 30px;

res/themes/legacy-dark/css/_legacy-dark.scss

@@ -204,6 +204,8 @@ $message-body-panel-fg-color: $primary-fg-color;
 // Appearance tab colors
 $appearance-tab-border-color: $room-highlight-color;
 
+$invite-reason-border-color: $room-highlight-color;
+
 $composer-shadow-color: tranparent;
 
 // ***** Mixins! *****

res/themes/legacy-light/css/_legacy-light.scss

@@ -333,6 +333,8 @@ $message-body-panel-fg-color: $muted-fg-color;
 // FontSlider colors
 $appearance-tab-border-color: $input-darker-bg-color;
 
+$invite-reason-border-color: $input-darker-bg-color;
+
 $composer-shadow-color: tranparent;
 
 // ***** Mixins! *****

res/themes/light/css/_light.scss

@@ -331,6 +331,8 @@ $message-body-panel-fg-color: $muted-fg-color;
 // FontSlider colors
 $appearance-tab-border-color: $input-darker-bg-color;
 
+$invite-reason-border-color: $input-darker-bg-color;
+
 // blur amounts for left left panel (only for element theme, used in _mods.scss)
 $roomlist-background-blur-amount: 40px;
 $groupFilterPanel-background-blur-amount: 20px;

src/CallHandler.tsx

@@ -154,6 +154,9 @@ function getRemoteAudioElement(): HTMLAudioElement {
 
 export default class CallHandler {
     private calls = new Map<string, MatrixCall>(); // roomId -> call
+    // Calls started as an attended transfer, ie. with the intention of transferring another
+    // call with a different party to this one.
+    private transferees = new Map<string, MatrixCall>(); // callId (target) -> call (transferee)
     private audioPromises = new Map<AudioID, Promise<void>>();
     private dispatcherRef: string = null;
     private supportsPstnProtocol = null;
@@ -325,6 +328,10 @@ export default class CallHandler {
         return callsNotInThatRoom;
     }
 
+    getTransfereeForCallId(callId: string): MatrixCall {
+        return this.transferees[callId];
+    }
+
     play(audioId: AudioID) {
         // TODO: Attach an invisible element for this instead
         // which listens?
@@ -622,6 +629,7 @@ export default class CallHandler {
     private async placeCall(
         roomId: string, type: PlaceCallType,
         localElement: HTMLVideoElement, remoteElement: HTMLVideoElement,
+        transferee: MatrixCall,
     ) {
         Analytics.trackEvent('voip', 'placeCall', 'type', type);
         CountlyAnalytics.instance.trackStartCall(roomId, type === PlaceCallType.Video, false);
@@ -634,6 +642,9 @@ export default class CallHandler {
         const call = createNewMatrixCall(MatrixClientPeg.get(), mappedRoomId);
 
         this.calls.set(roomId, call);
+        if (transferee) {
+            this.transferees[call.callId] = transferee;
+        }
 
         this.setCallListeners(call);
         this.setCallAudioElement(call);
@@ -723,7 +734,10 @@ export default class CallHandler {
                     } else if (members.length === 2) {
                         console.info(`Place ${payload.type} call in ${payload.room_id}`);
 
-                        this.placeCall(payload.room_id, payload.type, payload.local_element, payload.remote_element);
+                        this.placeCall(
+                            payload.room_id, payload.type, payload.local_element, payload.remote_element,
+                            payload.transferee,
+                        );
                     } else { // > 2
                         dis.dispatch({
                             action: "place_conference_call",

src/KeyBindingsDefaults.ts

@@ -161,27 +161,27 @@ const messageComposerBindings = (): KeyBinding<MessageComposerAction>[] => {
 const autocompleteBindings = (): KeyBinding<AutocompleteAction>[] => {
     return [
         {
-            action: AutocompleteAction.ApplySelection,
+            action: AutocompleteAction.CompleteOrNextSelection,
             keyCombo: {
                 key: Key.TAB,
             },
         },
         {
-            action: AutocompleteAction.ApplySelection,
+            action: AutocompleteAction.CompleteOrNextSelection,
             keyCombo: {
                 key: Key.TAB,
                 ctrlKey: true,
             },
         },
         {
-            action: AutocompleteAction.ApplySelection,
+            action: AutocompleteAction.CompleteOrPrevSelection,
             keyCombo: {
                 key: Key.TAB,
                 shiftKey: true,
             },
         },
         {
-            action: AutocompleteAction.ApplySelection,
+            action: AutocompleteAction.CompleteOrPrevSelection,
             keyCombo: {
                 key: Key.TAB,
                 ctrlKey: true,

src/KeyBindingsManager.ts

@@ -52,14 +52,19 @@ export enum MessageComposerAction {
 
 /** Actions for text editing autocompletion */
 export enum AutocompleteAction {
-    /** Apply the current autocomplete selection */
+    /**
-    ApplySelection = 'ApplySelection',
+     * Select previous selection or, if the autocompletion window is not shown, open the window and select the first
-    /** Cancel autocompletion */
+     * selection.
-    Cancel = 'Cancel',
+     */
+    CompleteOrPrevSelection = 'ApplySelection',
+    /** Select next selection or, if the autocompletion window is not shown, open it and select the first selection */
+    CompleteOrNextSelection = 'CompleteOrNextSelection',
     /** Move to the previous autocomplete selection */
     PrevSelection = 'PrevSelection',
     /** Move to the next autocomplete selection */
     NextSelection = 'NextSelection',
+    /** Close the autocompletion window */
+    Cancel = 'Cancel',
 }
 
 /** Actions for the room list sidebar */

src/autocomplete/QueryMatcher.ts

@@ -23,7 +23,6 @@ interface IOptions<T extends {}> {
     keys: Array<string | keyof T>;
     funcs?: Array<(T) => string>;
     shouldMatchWordsOnly?: boolean;
-    shouldMatchPrefix?: boolean;
     // whether to apply unhomoglyph and strip diacritics to fuzz up the search. Defaults to true
     fuzzy?: boolean;
 }
@@ -56,12 +55,6 @@ export default class QueryMatcher<T extends Object> {
         if (this._options.shouldMatchWordsOnly === undefined) {
             this._options.shouldMatchWordsOnly = true;
         }
-
-        // By default, match anywhere in the string being searched. If enabled, only return
-        // matches that are prefixed with the query.
-        if (this._options.shouldMatchPrefix === undefined) {
-            this._options.shouldMatchPrefix = false;
-        }
     }
 
     setObjects(objects: T[]) {
@@ -112,7 +105,7 @@ export default class QueryMatcher<T extends Object> {
                 resultKey = resultKey.replace(/[^\w]/g, '');
             }
             const index = resultKey.indexOf(query);
-            if (index !== -1 && (!this._options.shouldMatchPrefix || index === 0)) {
+            if (index !== -1) {
                 matches.push(
                     ...candidates.map((candidate) => ({index, ...candidate})),
                 );

src/autocomplete/UserProvider.tsx

@@ -56,7 +56,6 @@ export default class UserProvider extends AutocompleteProvider {
         this.matcher = new QueryMatcher([], {
             keys: ['name'],
             funcs: [obj => obj.userId.slice(1)], // index by user id minus the leading '@'
-            shouldMatchPrefix: true,
             shouldMatchWordsOnly: false,
         });
 

src/components/structures/GroupView.js

@@ -981,7 +981,7 @@ export default class GroupView extends React.Component {
                     <Spinner />
                 </div>;
             }
-            const httpInviterAvatar = this.state.inviterProfile
+            const httpInviterAvatar = this.state.inviterProfile && this.state.inviterProfile.avatarUrl
                 ? mediaFromMxc(this.state.inviterProfile.avatarUrl).getSquareThumbnailHttp(36)
                 : null;
 

src/components/structures/LeftPanel.tsx

@@ -34,7 +34,6 @@ import { UPDATE_EVENT } from "../../stores/AsyncStore";
 import ResizeNotifier from "../../utils/ResizeNotifier";
 import SettingsStore from "../../settings/SettingsStore";
 import RoomListStore, { LISTS_UPDATE_EVENT } from "../../stores/room-list/RoomListStore";
-import {Key} from "../../Keyboard";
 import IndicatorScrollbar from "../structures/IndicatorScrollbar";
 import AccessibleTooltipButton from "../views/elements/AccessibleTooltipButton";
 import { OwnProfileStore } from "../../stores/OwnProfileStore";
@@ -43,6 +42,7 @@ import LeftPanelWidget from "./LeftPanelWidget";
 import {replaceableComponent} from "../../utils/replaceableComponent";
 import {mediaFromMxc} from "../../customisations/Media";
 import SpaceStore, {UPDATE_SELECTED_SPACE} from "../../stores/SpaceStore";
+import { getKeyBindingsManager, RoomListAction } from "../../KeyBindingsManager";
 
 interface IProps {
     isMinimized: boolean;
@@ -297,17 +297,18 @@ export default class LeftPanel extends React.Component<IProps, IState> {
     private onKeyDown = (ev: React.KeyboardEvent) => {
         if (!this.focusedElement) return;
 
-        switch (ev.key) {
+        const action = getKeyBindingsManager().getRoomListAction(ev);
-            case Key.ARROW_UP:
+        switch (action) {
-            case Key.ARROW_DOWN:
+            case RoomListAction.NextRoom:
+            case RoomListAction.PrevRoom:
                 ev.stopPropagation();
                 ev.preventDefault();
-                this.onMoveFocus(ev.key === Key.ARROW_UP);
+                this.onMoveFocus(action === RoomListAction.PrevRoom);
                 break;
         }
     };
 
-    private onEnter = () => {
+    private selectRoom = () => {
         const firstRoom = this.listContainerRef.current.querySelector<HTMLDivElement>(".mx_RoomTile");
         if (firstRoom) {
             firstRoom.click();
@@ -388,8 +389,8 @@ export default class LeftPanel extends React.Component<IProps, IState> {
             >
                 <RoomSearch
                     isMinimized={this.props.isMinimized}
-                    onVerticalArrow={this.onKeyDown}
+                    onKeyDown={this.onKeyDown}
-                    onEnter={this.onEnter}
+                    onSelectRoom={this.selectRoom}
                 />
                 <AccessibleTooltipButton
                     className={classNames("mx_LeftPanel_exploreButton", {

src/components/structures/LoggedInView.tsx

@@ -444,6 +444,7 @@ class LoggedInView extends React.Component<IProps, IState> {
             case RoomAction.RoomScrollDown:
             case RoomAction.JumpToFirstMessage:
             case RoomAction.JumpToLatestMessage:
+                // pass the event down to the scroll panel
                 this._onScrollKeyPressed(ev);
                 handled = true;
                 break;

src/components/structures/RoomSearch.tsx

@@ -30,8 +30,11 @@ import SpaceStore, {UPDATE_SELECTED_SPACE} from "../../stores/SpaceStore";
 
 interface IProps {
     isMinimized: boolean;
-    onVerticalArrow(ev: React.KeyboardEvent): void;
+    onKeyDown(ev: React.KeyboardEvent): void;
-    onEnter(ev: React.KeyboardEvent): boolean;
+    /**
+     * @returns true if a room has been selected and the search field should be cleared
+     */
+    onSelectRoom(): boolean;
 }
 
 interface IState {
@@ -120,10 +123,11 @@ export default class RoomSearch extends React.PureComponent<IProps, IState> {
                 break;
             case RoomListAction.NextRoom:
             case RoomListAction.PrevRoom:
-                this.props.onVerticalArrow(ev);
+                // we don't handle these actions here put pass the event on to the interested party (LeftPanel)
+                this.props.onKeyDown(ev);
                 break;
             case RoomListAction.SelectRoom: {
-                const shouldClear = this.props.onEnter(ev);
+                const shouldClear = this.props.onSelectRoom();
                 if (shouldClear) {
                     // wrap in set immediate to delay it so that we don't clear the filter & then change room
                     setImmediate(() => {

src/components/structures/ScrollPanel.js

@@ -16,10 +16,10 @@ limitations under the License.
 
 import React, {createRef} from "react";
 import PropTypes from 'prop-types';
-import { Key } from '../../Keyboard';
 import Timer from '../../utils/Timer';
 import AutoHideScrollbar from "./AutoHideScrollbar";
 import {replaceableComponent} from "../../utils/replaceableComponent";
+import {getKeyBindingsManager, RoomAction} from "../../KeyBindingsManager";
 
 const DEBUG_SCROLL = false;
 
@@ -539,34 +539,24 @@ export default class ScrollPanel extends React.Component {
      * @param {object} ev the keyboard event
      */
     handleScrollKey = ev => {
-        let isScrolling = false;
+	let isScrolling = false;
-        switch (ev.key) {
+        const roomAction = getKeyBindingsManager().getRoomAction(ev);
-            case Key.PAGE_UP:
+        switch (roomAction) {
-                if (!ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+            case RoomAction.ScrollUp:
-                    isScrolling = true;
+                this.scrollRelative(-1);
-                    this.scrollRelative(-1);
+		isScrolling = true;
-                }
                 break;
-
+            case RoomAction.RoomScrollDown:
-            case Key.PAGE_DOWN:
+                this.scrollRelative(1);
-                if (!ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+		isScrolling = true;
-                    isScrolling = true;
-                    this.scrollRelative(1);
-                }
                 break;
-
+            case RoomAction.JumpToFirstMessage:
-            case Key.HOME:
+                this.scrollToTop();
-                if (ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+		isScrolling = true;
-                    isScrolling = true;
-                    this.scrollToTop();
-                }
                 break;
-
+            case RoomAction.JumpToLatestMessage:
-            case Key.END:
+                this.scrollToBottom();
-                if (ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+		isScrolling = true;
-                    isScrolling = true;
-                    this.scrollToBottom();
-                }
                 break;
         }
         if (isScrolling && this.props.onUserScroll) {

src/components/structures/ScrollPanel.js.orig

@@ -0,0 +1,938 @@
+/*
+Copyright 2015, 2016 OpenMarket Ltd
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+import React, {createRef} from "react";
+import PropTypes from 'prop-types';
+import Timer from '../../utils/Timer';
+import AutoHideScrollbar from "./AutoHideScrollbar";
+import {replaceableComponent} from "../../utils/replaceableComponent";
+import {getKeyBindingsManager, RoomAction} from "../../KeyBindingsManager";
+
+const DEBUG_SCROLL = false;
+
+// The amount of extra scroll distance to allow prior to unfilling.
+// See _getExcessHeight.
+const UNPAGINATION_PADDING = 6000;
+// The number of milliseconds to debounce calls to onUnfillRequest, to prevent
+// many scroll events causing many unfilling requests.
+const UNFILL_REQUEST_DEBOUNCE_MS = 200;
+// _updateHeight makes the height a ceiled multiple of this so we
+// don't have to update the height too often. It also allows the user
+// to scroll past the pagination spinner a bit so they don't feel blocked so
+// much while the content loads.
+const PAGE_SIZE = 400;
+
+let debuglog;
+if (DEBUG_SCROLL) {
+    // using bind means that we get to keep useful line numbers in the console
+    debuglog = console.log.bind(console, "ScrollPanel debuglog:");
+} else {
+    debuglog = function() {};
+}
+
+/* This component implements an intelligent scrolling list.
+ *
+ * It wraps a list of <li> children; when items are added to the start or end
+ * of the list, the scroll position is updated so that the user still sees the
+ * same position in the list.
+ *
+ * It also provides a hook which allows parents to provide more list elements
+ * when we get close to the start or end of the list.
+ *
+ * Each child element should have a 'data-scroll-tokens'. This string of
+ * comma-separated tokens may contain a single token or many, where many indicates
+ * that the element contains elements that have scroll tokens themselves. The first
+ * token in 'data-scroll-tokens' is used to serialise the scroll state, and returned
+ * as the 'trackedScrollToken' attribute by getScrollState().
+ *
+ * IMPORTANT: INDIVIDUAL TOKENS WITHIN 'data-scroll-tokens' MUST NOT CONTAIN COMMAS.
+ *
+ * Some notes about the implementation:
+ *
+ * The saved 'scrollState' can exist in one of two states:
+ *
+ *   - stuckAtBottom: (the default, and restored by resetScrollState): the
+ *     viewport is scrolled down as far as it can be. When the children are
+ *     updated, the scroll position will be updated to ensure it is still at
+ *     the bottom.
+ *
+ *   - fixed, in which the viewport is conceptually tied at a specific scroll
+ *     offset.  We don't save the absolute scroll offset, because that would be
+ *     affected by window width, zoom level, amount of scrollback, etc. Instead
+ *     we save an identifier for the last fully-visible message, and the number
+ *     of pixels the window was scrolled below it - which is hopefully near
+ *     enough.
+ *
+ * The 'stickyBottom' property controls the behaviour when we reach the bottom
+ * of the window (either through a user-initiated scroll, or by calling
+ * scrollToBottom). If stickyBottom is enabled, the scrollState will enter
+ * 'stuckAtBottom' state - ensuring that new additions cause the window to
+ * scroll down further. If stickyBottom is disabled, we just save the scroll
+ * offset as normal.
+ */
+
+@replaceableComponent("structures.ScrollPanel")
+export default class ScrollPanel extends React.Component {
+    static propTypes = {
+        /* stickyBottom: if set to true, then once the user hits the bottom of
+         * the list, any new children added to the list will cause the list to
+         * scroll down to show the new element, rather than preserving the
+         * existing view.
+         */
+        stickyBottom: PropTypes.bool,
+
+        /* startAtBottom: if set to true, the view is assumed to start
+         * scrolled to the bottom.
+         * XXX: It's likely this is unnecessary and can be derived from
+         * stickyBottom, but I'm adding an extra parameter to ensure
+         * behaviour stays the same for other uses of ScrollPanel.
+         * If so, let's remove this parameter down the line.
+         */
+        startAtBottom: PropTypes.bool,
+
+        /* onFillRequest(backwards): a callback which is called on scroll when
+         * the user nears the start (backwards = true) or end (backwards =
+         * false) of the list.
+         *
+         * This should return a promise; no more calls will be made until the
+         * promise completes.
+         *
+         * The promise should resolve to true if there is more data to be
+         * retrieved in this direction (in which case onFillRequest may be
+         * called again immediately), or false if there is no more data in this
+         * directon (at this time) - which will stop the pagination cycle until
+         * the user scrolls again.
+         */
+        onFillRequest: PropTypes.func,
+
+        /* onUnfillRequest(backwards): a callback which is called on scroll when
+         * there are children elements that are far out of view and could be removed
+         * without causing pagination to occur.
+         *
+         * This function should accept a boolean, which is true to indicate the back/top
+         * of the panel and false otherwise, and a scroll token, which refers to the
+         * first element to remove if removing from the front/bottom, and last element
+         * to remove if removing from the back/top.
+         */
+        onUnfillRequest: PropTypes.func,
+
+        /* onScroll: a callback which is called whenever any scroll happens.
+         */
+        onScroll: PropTypes.func,
+
+        /* onUserScroll: callback which is called when the user interacts with the room timeline
+         */
+        onUserScroll: PropTypes.func,
+
+        /* className: classnames to add to the top-level div
+         */
+        className: PropTypes.string,
+
+        /* style: styles to add to the top-level div
+         */
+        style: PropTypes.object,
+
+        /* resizeNotifier: ResizeNotifier to know when middle column has changed size
+         */
+        resizeNotifier: PropTypes.object,
+
+        /* fixedChildren: allows for children to be passed which are rendered outside
+         * of the wrapper
+         */
+        fixedChildren: PropTypes.node,
+    };
+
+    static defaultProps = {
+        stickyBottom: true,
+        startAtBottom: true,
+        onFillRequest: function(backwards) { return Promise.resolve(false); },
+        onUnfillRequest: function(backwards, scrollToken) {},
+        onScroll: function() {},
+    };
+
+    constructor(props) {
+        super(props);
+
+        this._pendingFillRequests = {b: null, f: null};
+
+        if (this.props.resizeNotifier) {
+            this.props.resizeNotifier.on("middlePanelResizedNoisy", this.onResize);
+        }
+
+        this.resetScrollState();
+
+        this._itemlist = createRef();
+    }
+
+    componentDidMount() {
+        this.checkScroll();
+    }
+
+    componentDidUpdate() {
+        // after adding event tiles, we may need to tweak the scroll (either to
+        // keep at the bottom of the timeline, or to maintain the view after
+        // adding events to the top).
+        //
+        // This will also re-check the fill state, in case the paginate was inadequate
+        this.checkScroll();
+        this.updatePreventShrinking();
+    }
+
+    componentWillUnmount() {
+        // set a boolean to say we've been unmounted, which any pending
+        // promises can use to throw away their results.
+        //
+        // (We could use isMounted(), but facebook have deprecated that.)
+        this.unmounted = true;
+
+        if (this.props.resizeNotifier) {
+            this.props.resizeNotifier.removeListener("middlePanelResizedNoisy", this.onResize);
+        }
+    }
+
+    onScroll = ev => {
+        // skip scroll events caused by resizing
+        if (this.props.resizeNotifier && this.props.resizeNotifier.isResizing) return;
+        debuglog("onScroll", this._getScrollNode().scrollTop);
+        this._scrollTimeout.restart();
+        this._saveScrollState();
+        this.updatePreventShrinking();
+        this.props.onScroll(ev);
+        this.checkFillState();
+    };
+
+    onResize = () => {
+        debuglog("onResize");
+        this.checkScroll();
+        // update preventShrinkingState if present
+        if (this.preventShrinkingState) {
+            this.preventShrinking();
+        }
+    };
+
+    // after an update to the contents of the panel, check that the scroll is
+    // where it ought to be, and set off pagination requests if necessary.
+    checkScroll = () => {
+        if (this.unmounted) {
+            return;
+        }
+        this._restoreSavedScrollState();
+        this.checkFillState();
+    };
+
+    // return true if the content is fully scrolled down right now; else false.
+    //
+    // note that this is independent of the 'stuckAtBottom' state - it is simply
+    // about whether the content is scrolled down right now, irrespective of
+    // whether it will stay that way when the children update.
+    isAtBottom = () => {
+        const sn = this._getScrollNode();
+        // fractional values (both too big and too small)
+        // for scrollTop happen on certain browsers/platforms
+        // when scrolled all the way down. E.g. Chrome 72 on debian.
+        // so check difference <= 1;
+        return Math.abs(sn.scrollHeight - (sn.scrollTop + sn.clientHeight)) <= 1;
+    };
+
+    // returns the vertical height in the given direction that can be removed from
+    // the content box (which has a height of scrollHeight, see checkFillState) without
+    // pagination occuring.
+    //
+    // padding* = UNPAGINATION_PADDING
+    //
+    // ### Region determined as excess.
+    //
+    //   .---------.                        -              -
+    //   |#########|                        |              |
+    //   |#########|   -                    |  scrollTop   |
+    //   |         |   | padding*           |              |
+    //   |         |   |                    |              |
+    // .-+---------+-. -  -                 |              |
+    // : |         | :    |                 |              |
+    // : |         | :    |  clientHeight   |              |
+    // : |         | :    |                 |              |
+    // .-+---------+-.    -                 -              |
+    // | |         | |    |                                |
+    // | |         | |    |  clientHeight                  | scrollHeight
+    // | |         | |    |                                |
+    // `-+---------+-'    -                                |
+    // : |         | :    |                                |
+    // : |         | :    |  clientHeight                  |
+    // : |         | :    |                                |
+    // `-+---------+-' -  -                                |
+    //   |         |   | padding*                          |
+    //   |         |   |                                   |
+    //   |#########|   -                                   |
+    //   |#########|                                       |
+    //   `---------'                                       -
+    _getExcessHeight(backwards) {
+        const sn = this._getScrollNode();
+        const contentHeight = this._getMessagesHeight();
+        const listHeight = this._getListHeight();
+        const clippedHeight = contentHeight - listHeight;
+        const unclippedScrollTop = sn.scrollTop + clippedHeight;
+
+        if (backwards) {
+            return unclippedScrollTop - sn.clientHeight - UNPAGINATION_PADDING;
+        } else {
+            return contentHeight - (unclippedScrollTop + 2*sn.clientHeight) - UNPAGINATION_PADDING;
+        }
+    }
+
+    // check the scroll state and send out backfill requests if necessary.
+    checkFillState = async (depth=0) => {
+        if (this.unmounted) {
+            return;
+        }
+
+        const isFirstCall = depth === 0;
+        const sn = this._getScrollNode();
+
+        // if there is less than a screenful of messages above or below the
+        // viewport, try to get some more messages.
+        //
+        // scrollTop is the number of pixels between the top of the content and
+        //     the top of the viewport.
+        //
+        // scrollHeight is the total height of the content.
+        //
+        // clientHeight is the height of the viewport (excluding borders,
+        // margins, and scrollbars).
+        //
+        //
+        //   .---------.          -                 -
+        //   |         |          |  scrollTop      |
+        // .-+---------+-.    -   -                 |
+        // | |         | |    |                     |
+        // | |         | |    |  clientHeight       | scrollHeight
+        // | |         | |    |                     |
+        // `-+---------+-'    -                     |
+        //   |         |                            |
+        //   |         |                            |
+        //   `---------'                            -
+        //
+
+        // as filling is async and recursive,
+        // don't allow more than 1 chain of calls concurrently
+        // do make a note when a new request comes in while already running one,
+        // so we can trigger a new chain of calls once done.
+        if (isFirstCall) {
+            if (this._isFilling) {
+                debuglog("_isFilling: not entering while request is ongoing, marking for a subsequent request");
+                this._fillRequestWhileRunning = true;
+                return;
+            }
+            debuglog("_isFilling: setting");
+            this._isFilling = true;
+        }
+
+        const itemlist = this._itemlist.current;
+        const firstTile = itemlist && itemlist.firstElementChild;
+        const contentTop = firstTile && firstTile.offsetTop;
+        const fillPromises = [];
+
+        // if scrollTop gets to 1 screen from the top of the first tile,
+        // try backward filling
+        if (!firstTile || (sn.scrollTop - contentTop) < sn.clientHeight) {
+            // need to back-fill
+            fillPromises.push(this._maybeFill(depth, true));
+        }
+        // if scrollTop gets to 2 screens from the end (so 1 screen below viewport),
+        // try forward filling
+        if ((sn.scrollHeight - sn.scrollTop) < sn.clientHeight * 2) {
+            // need to forward-fill
+            fillPromises.push(this._maybeFill(depth, false));
+        }
+
+        if (fillPromises.length) {
+            try {
+                await Promise.all(fillPromises);
+            } catch (err) {
+                console.error(err);
+            }
+        }
+        if (isFirstCall) {
+            debuglog("_isFilling: clearing");
+            this._isFilling = false;
+        }
+
+        if (this._fillRequestWhileRunning) {
+            this._fillRequestWhileRunning = false;
+            this.checkFillState();
+        }
+    };
+
+    // check if unfilling is possible and send an unfill request if necessary
+    _checkUnfillState(backwards) {
+        let excessHeight = this._getExcessHeight(backwards);
+        if (excessHeight <= 0) {
+            return;
+        }
+
+        const origExcessHeight = excessHeight;
+
+        const tiles = this._itemlist.current.children;
+
+        // The scroll token of the first/last tile to be unpaginated
+        let markerScrollToken = null;
+
+        // Subtract heights of tiles to simulate the tiles being unpaginated until the
+        // excess height is less than the height of the next tile to subtract. This
+        // prevents excessHeight becoming negative, which could lead to future
+        // pagination.
+        //
+        // If backwards is true, we unpaginate (remove) tiles from the back (top).
+        let tile;
+        for (let i = 0; i < tiles.length; i++) {
+            tile = tiles[backwards ? i : tiles.length - 1 - i];
+            // Subtract height of tile as if it were unpaginated
+            excessHeight -= tile.clientHeight;
+            //If removing the tile would lead to future pagination, break before setting scroll token
+            if (tile.clientHeight > excessHeight) {
+                break;
+            }
+            // The tile may not have a scroll token, so guard it
+            if (tile.dataset.scrollTokens) {
+                markerScrollToken = tile.dataset.scrollTokens.split(',')[0];
+            }
+        }
+
+        if (markerScrollToken) {
+            // Use a debouncer to prevent multiple unfill calls in quick succession
+            // This is to make the unfilling process less aggressive
+            if (this._unfillDebouncer) {
+                clearTimeout(this._unfillDebouncer);
+            }
+            this._unfillDebouncer = setTimeout(() => {
+                this._unfillDebouncer = null;
+                debuglog("unfilling now", backwards, origExcessHeight);
+                this.props.onUnfillRequest(backwards, markerScrollToken);
+            }, UNFILL_REQUEST_DEBOUNCE_MS);
+        }
+    }
+
+    // check if there is already a pending fill request. If not, set one off.
+    _maybeFill(depth, backwards) {
+        const dir = backwards ? 'b' : 'f';
+        if (this._pendingFillRequests[dir]) {
+            debuglog("Already a "+dir+" fill in progress - not starting another");
+            return;
+        }
+
+        debuglog("starting "+dir+" fill");
+
+        // onFillRequest can end up calling us recursively (via onScroll
+        // events) so make sure we set this before firing off the call.
+        this._pendingFillRequests[dir] = true;
+
+        // wait 1ms before paginating, because otherwise
+        // this will block the scroll event handler for +700ms
+        // if messages are already cached in memory,
+        // This would cause jumping to happen on Chrome/macOS.
+        return new Promise(resolve => setTimeout(resolve, 1)).then(() => {
+            return this.props.onFillRequest(backwards);
+        }).finally(() => {
+            this._pendingFillRequests[dir] = false;
+        }).then((hasMoreResults) => {
+            if (this.unmounted) {
+                return;
+            }
+            // Unpaginate once filling is complete
+            this._checkUnfillState(!backwards);
+
+            debuglog(""+dir+" fill complete; hasMoreResults:"+hasMoreResults);
+            if (hasMoreResults) {
+                // further pagination requests have been disabled until now, so
+                // it's time to check the fill state again in case the pagination
+                // was insufficient.
+                return this.checkFillState(depth + 1);
+            }
+        });
+    }
+
+    /* get the current scroll state. This returns an object with the following
+     * properties:
+     *
+     * boolean stuckAtBottom: true if we are tracking the bottom of the
+     *   scroll. false if we are tracking a particular child.
+     *
+     * string trackedScrollToken: undefined if stuckAtBottom is true; if it is
+     *   false, the first token in data-scroll-tokens of the child which we are
+     *   tracking.
+     *
+     * number bottomOffset: undefined if stuckAtBottom is true; if it is false,
+     *   the number of pixels the bottom of the tracked child is above the
+     *   bottom of the scroll panel.
+     */
+    getScrollState = () => this.scrollState;
+
+    /* reset the saved scroll state.
+     *
+     * This is useful if the list is being replaced, and you don't want to
+     * preserve scroll even if new children happen to have the same scroll
+     * tokens as old ones.
+     *
+     * This will cause the viewport to be scrolled down to the bottom on the
+     * next update of the child list. This is different to scrollToBottom(),
+     * which would save the current bottom-most child as the active one (so is
+     * no use if no children exist yet, or if you are about to replace the
+     * child list.)
+     */
+    resetScrollState = () => {
+        this.scrollState = {
+            stuckAtBottom: this.props.startAtBottom,
+        };
+        this._bottomGrowth = 0;
+        this._pages = 0;
+        this._scrollTimeout = new Timer(100);
+        this._heightUpdateInProgress = false;
+    };
+
+    /**
+     * jump to the top of the content.
+     */
+    scrollToTop = () => {
+        this._getScrollNode().scrollTop = 0;
+        this._saveScrollState();
+    };
+
+    /**
+     * jump to the bottom of the content.
+     */
+    scrollToBottom = () => {
+        // the easiest way to make sure that the scroll state is correctly
+        // saved is to do the scroll, then save the updated state. (Calculating
+        // it ourselves is hard, and we can't rely on an onScroll callback
+        // happening, since there may be no user-visible change here).
+        const sn = this._getScrollNode();
+        sn.scrollTop = sn.scrollHeight;
+        this._saveScrollState();
+    };
+
+    /**
+     * Page up/down.
+     *
+     * @param {number} mult: -1 to page up, +1 to page down
+     */
+    scrollRelative = mult => {
+        const scrollNode = this._getScrollNode();
+        const delta = mult * scrollNode.clientHeight * 0.5;
+        scrollNode.scrollBy(0, delta);
+        this._saveScrollState();
+    };
+
+    /**
+     * Scroll up/down in response to a scroll key
+     * @param {object} ev the keyboard event
+     */
+    handleScrollKey = ev => {
+<<<<<<< HEAD
+        let isScrolling = false;
+        switch (ev.key) {
+            case Key.PAGE_UP:
+                if (!ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+                    isScrolling = true;
+                    this.scrollRelative(-1);
+                }
+                break;
+
+            case Key.PAGE_DOWN:
+                if (!ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+                    isScrolling = true;
+                    this.scrollRelative(1);
+                }
+                break;
+
+            case Key.HOME:
+                if (ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+                    isScrolling = true;
+                    this.scrollToTop();
+                }
+                break;
+
+            case Key.END:
+                if (ev.ctrlKey && !ev.shiftKey && !ev.altKey && !ev.metaKey) {
+                    isScrolling = true;
+                    this.scrollToBottom();
+                }
+=======
+        const roomAction = getKeyBindingsManager().getRoomAction(ev);
+        switch (roomAction) {
+            case RoomAction.ScrollUp:
+                this.scrollRelative(-1);
+                break;
+            case RoomAction.RoomScrollDown:
+                this.scrollRelative(1);
+                break;
+            case RoomAction.JumpToFirstMessage:
+                this.scrollToTop();
+                break;
+            case RoomAction.JumpToLatestMessage:
+                this.scrollToBottom();
+>>>>>>> develop
+                break;
+        }
+        if (isScrolling && this.props.onUserScroll) {
+            this.props.onUserScroll(ev);
+        }
+    };
+
+    /* Scroll the panel to bring the DOM node with the scroll token
+     * `scrollToken` into view.
+     *
+     * offsetBase gives the reference point for the pixelOffset. 0 means the
+     * top of the container, 1 means the bottom, and fractional values mean
+     * somewhere in the middle. If omitted, it defaults to 0.
+     *
+     * pixelOffset gives the number of pixels *above* the offsetBase that the
+     * node (specifically, the bottom of it) will be positioned. If omitted, it
+     * defaults to 0.
+     */
+    scrollToToken = (scrollToken, pixelOffset, offsetBase) => {
+        pixelOffset = pixelOffset || 0;
+        offsetBase = offsetBase || 0;
+
+        // set the trackedScrollToken so we can get the node through _getTrackedNode
+        this.scrollState = {
+            stuckAtBottom: false,
+            trackedScrollToken: scrollToken,
+        };
+        const trackedNode = this._getTrackedNode();
+        const scrollNode = this._getScrollNode();
+        if (trackedNode) {
+            // set the scrollTop to the position we want.
+            // note though, that this might not succeed if the combination of offsetBase and pixelOffset
+            // would position the trackedNode towards the top of the viewport.
+            // This because when setting the scrollTop only 10 or so events might be loaded,
+            // not giving enough content below the trackedNode to scroll downwards
+            // enough so it ends up in the top of the viewport.
+            debuglog("scrollToken: setting scrollTop", {offsetBase, pixelOffset, offsetTop: trackedNode.offsetTop});
+            scrollNode.scrollTop = (trackedNode.offsetTop - (scrollNode.clientHeight * offsetBase)) + pixelOffset;
+            this._saveScrollState();
+        }
+    };
+
+    _saveScrollState() {
+        if (this.props.stickyBottom && this.isAtBottom()) {
+            this.scrollState = { stuckAtBottom: true };
+            debuglog("saved stuckAtBottom state");
+            return;
+        }
+
+        const scrollNode = this._getScrollNode();
+        const viewportBottom = scrollNode.scrollHeight - (scrollNode.scrollTop + scrollNode.clientHeight);
+
+        const itemlist = this._itemlist.current;
+        const messages = itemlist.children;
+        let node = null;
+
+        // TODO: do a binary search here, as items are sorted by offsetTop
+        // loop backwards, from bottom-most message (as that is the most common case)
+        for (let i = messages.length-1; i >= 0; --i) {
+            if (!messages[i].dataset.scrollTokens) {
+                continue;
+            }
+            node = messages[i];
+            // break at the first message (coming from the bottom)
+            // that has it's offsetTop above the bottom of the viewport.
+            if (this._topFromBottom(node) > viewportBottom) {
+                // Use this node as the scrollToken
+                break;
+            }
+        }
+
+        if (!node) {
+            debuglog("unable to save scroll state: found no children in the viewport");
+            return;
+        }
+        const scrollToken = node.dataset.scrollTokens.split(',')[0];
+        debuglog("saving anchored scroll state to message", node && node.innerText, scrollToken);
+        const bottomOffset = this._topFromBottom(node);
+        this.scrollState = {
+            stuckAtBottom: false,
+            trackedNode: node,
+            trackedScrollToken: scrollToken,
+            bottomOffset: bottomOffset,
+            pixelOffset: bottomOffset - viewportBottom, //needed for restoring the scroll position when coming back to the room
+        };
+    }
+
+    async _restoreSavedScrollState() {
+        const scrollState = this.scrollState;
+
+        if (scrollState.stuckAtBottom) {
+            const sn = this._getScrollNode();
+            if (sn.scrollTop !== sn.scrollHeight) {
+                sn.scrollTop = sn.scrollHeight;
+            }
+        } else if (scrollState.trackedScrollToken) {
+            const itemlist = this._itemlist.current;
+            const trackedNode = this._getTrackedNode();
+            if (trackedNode) {
+                const newBottomOffset = this._topFromBottom(trackedNode);
+                const bottomDiff = newBottomOffset - scrollState.bottomOffset;
+                this._bottomGrowth += bottomDiff;
+                scrollState.bottomOffset = newBottomOffset;
+                const newHeight = `${this._getListHeight()}px`;
+                if (itemlist.style.height !== newHeight) {
+                    itemlist.style.height = newHeight;
+                }
+                debuglog("balancing height because messages below viewport grew by", bottomDiff);
+            }
+        }
+        if (!this._heightUpdateInProgress) {
+            this._heightUpdateInProgress = true;
+            try {
+                await this._updateHeight();
+            } finally {
+                this._heightUpdateInProgress = false;
+            }
+        } else {
+            debuglog("not updating height because request already in progress");
+        }
+    }
+
+    // need a better name that also indicates this will change scrollTop? Rebalance height? Reveal content?
+    async _updateHeight() {
+        // wait until user has stopped scrolling
+        if (this._scrollTimeout.isRunning()) {
+            debuglog("updateHeight waiting for scrolling to end ... ");
+            await this._scrollTimeout.finished();
+        } else {
+            debuglog("updateHeight getting straight to business, no scrolling going on.");
+        }
+
+        // We might have unmounted since the timer finished, so abort if so.
+        if (this.unmounted) {
+            return;
+        }
+
+        const sn = this._getScrollNode();
+        const itemlist = this._itemlist.current;
+        const contentHeight = this._getMessagesHeight();
+        const minHeight = sn.clientHeight;
+        const height = Math.max(minHeight, contentHeight);
+        this._pages = Math.ceil(height / PAGE_SIZE);
+        this._bottomGrowth = 0;
+        const newHeight = `${this._getListHeight()}px`;
+
+        const scrollState = this.scrollState;
+        if (scrollState.stuckAtBottom) {
+            if (itemlist.style.height !== newHeight) {
+                itemlist.style.height = newHeight;
+            }
+            if (sn.scrollTop !== sn.scrollHeight) {
+                sn.scrollTop = sn.scrollHeight;
+            }
+            debuglog("updateHeight to", newHeight);
+        } else if (scrollState.trackedScrollToken) {
+            const trackedNode = this._getTrackedNode();
+            // if the timeline has been reloaded
+            // this can be called before scrollToBottom or whatever has been called
+            // so don't do anything if the node has disappeared from
+            // the currently filled piece of the timeline
+            if (trackedNode) {
+                const oldTop = trackedNode.offsetTop;
+                if (itemlist.style.height !== newHeight) {
+                    itemlist.style.height = newHeight;
+                }
+                const newTop = trackedNode.offsetTop;
+                const topDiff = newTop - oldTop;
+                // important to scroll by a relative amount as
+                // reading scrollTop and then setting it might
+                // yield out of date values and cause a jump
+                // when setting it
+                sn.scrollBy(0, topDiff);
+                debuglog("updateHeight to", {newHeight, topDiff});
+            }
+        }
+    }
+
+    _getTrackedNode() {
+        const scrollState = this.scrollState;
+        const trackedNode = scrollState.trackedNode;
+
+        if (!trackedNode || !trackedNode.parentElement) {
+            let node;
+            const messages = this._itemlist.current.children;
+            const scrollToken = scrollState.trackedScrollToken;
+
+            for (let i = messages.length-1; i >= 0; --i) {
+                const m = messages[i];
+                // 'data-scroll-tokens' is a DOMString of comma-separated scroll tokens
+                // There might only be one scroll token
+                if (m.dataset.scrollTokens &&
+                    m.dataset.scrollTokens.split(',').indexOf(scrollToken) !== -1) {
+                    node = m;
+                    break;
+                }
+            }
+            if (node) {
+                debuglog("had to find tracked node again for " + scrollState.trackedScrollToken);
+            }
+            scrollState.trackedNode = node;
+        }
+
+        if (!scrollState.trackedNode) {
+            debuglog("No node with ; '"+scrollState.trackedScrollToken+"'");
+            return;
+        }
+
+        return scrollState.trackedNode;
+    }
+
+    _getListHeight() {
+        return this._bottomGrowth + (this._pages * PAGE_SIZE);
+    }
+
+    _getMessagesHeight() {
+        const itemlist = this._itemlist.current;
+        const lastNode = itemlist.lastElementChild;
+        const lastNodeBottom = lastNode ? lastNode.offsetTop + lastNode.clientHeight : 0;
+        const firstNodeTop = itemlist.firstElementChild ? itemlist.firstElementChild.offsetTop : 0;
+        // 18 is itemlist padding
+        return lastNodeBottom - firstNodeTop + (18 * 2);
+    }
+
+    _topFromBottom(node) {
+        // current capped height - distance from top = distance from bottom of container to top of tracked element
+        return this._itemlist.current.clientHeight - node.offsetTop;
+    }
+
+    /* get the DOM node which has the scrollTop property we care about for our
+     * message panel.
+     */
+    _getScrollNode() {
+        if (this.unmounted) {
+            // this shouldn't happen, but when it does, turn the NPE into
+            // something more meaningful.
+            throw new Error("ScrollPanel._getScrollNode called when unmounted");
+        }
+
+        if (!this._divScroll) {
+            // Likewise, we should have the ref by this point, but if not
+            // turn the NPE into something meaningful.
+            throw new Error("ScrollPanel._getScrollNode called before AutoHideScrollbar ref collected");
+        }
+
+        return this._divScroll;
+    }
+
+    _collectScroll = divScroll => {
+        this._divScroll = divScroll;
+    };
+
+    /**
+    Mark the bottom offset of the last tile so we can balance it out when
+    anything below it changes, by calling updatePreventShrinking, to keep
+    the same minimum bottom offset, effectively preventing the timeline to shrink.
+    */
+    preventShrinking = () => {
+        const messageList = this._itemlist.current;
+        const tiles = messageList && messageList.children;
+        if (!messageList) {
+            return;
+        }
+        let lastTileNode;
+        for (let i = tiles.length - 1; i >= 0; i--) {
+            const node = tiles[i];
+            if (node.dataset.scrollTokens) {
+                lastTileNode = node;
+                break;
+            }
+        }
+        if (!lastTileNode) {
+            return;
+        }
+        this.clearPreventShrinking();
+        const offsetFromBottom = messageList.clientHeight - (lastTileNode.offsetTop + lastTileNode.clientHeight);
+        this.preventShrinkingState = {
+            offsetFromBottom: offsetFromBottom,
+            offsetNode: lastTileNode,
+        };
+        debuglog("prevent shrinking, last tile ", offsetFromBottom, "px from bottom");
+    };
+
+    /** Clear shrinking prevention. Used internally, and when the timeline is reloaded. */
+    clearPreventShrinking = () => {
+        const messageList = this._itemlist.current;
+        const balanceElement = messageList && messageList.parentElement;
+        if (balanceElement) balanceElement.style.paddingBottom = null;
+        this.preventShrinkingState = null;
+        debuglog("prevent shrinking cleared");
+    };
+
+    /**
+    update the container padding to balance
+    the bottom offset of the last tile since
+    preventShrinking was called.
+    Clears the prevent-shrinking state ones the offset
+    from the bottom of the marked tile grows larger than
+    what it was when marking.
+    */
+    updatePreventShrinking = () => {
+        if (this.preventShrinkingState) {
+            const sn = this._getScrollNode();
+            const scrollState = this.scrollState;
+            const messageList = this._itemlist.current;
+            const {offsetNode, offsetFromBottom} = this.preventShrinkingState;
+            // element used to set paddingBottom to balance the typing notifs disappearing
+            const balanceElement = messageList.parentElement;
+            // if the offsetNode got unmounted, clear
+            let shouldClear = !offsetNode.parentElement;
+            // also if 200px from bottom
+            if (!shouldClear && !scrollState.stuckAtBottom) {
+                const spaceBelowViewport = sn.scrollHeight - (sn.scrollTop + sn.clientHeight);
+                shouldClear = spaceBelowViewport >= 200;
+            }
+            // try updating if not clearing
+            if (!shouldClear) {
+                const currentOffset = messageList.clientHeight - (offsetNode.offsetTop + offsetNode.clientHeight);
+                const offsetDiff = offsetFromBottom - currentOffset;
+                if (offsetDiff > 0) {
+                    balanceElement.style.paddingBottom = `${offsetDiff}px`;
+                    debuglog("update prevent shrinking ", offsetDiff, "px from bottom");
+                } else if (offsetDiff < 0) {
+                    shouldClear = true;
+                }
+            }
+            if (shouldClear) {
+                this.clearPreventShrinking();
+            }
+        }
+    };
+
+    render() {
+        // TODO: the classnames on the div and ol could do with being updated to
+        // reflect the fact that we don't necessarily contain a list of messages.
+        // it's not obvious why we have a separate div and ol anyway.
+
+        // give the <ol> an explicit role=list because Safari+VoiceOver seems to think an ordered-list with
+        // list-style-type: none; is no longer a list
+        return (<AutoHideScrollbar wrappedRef={this._collectScroll}
+                onScroll={this.onScroll}
+                onWheel={this.props.onUserScroll}
+                className={`mx_ScrollPanel ${this.props.className}`} style={this.props.style}>
+                    { this.props.fixedChildren }
+                    <div className="mx_RoomView_messageListWrapper">
+                        <ol ref={this._itemlist} className="mx_RoomView_MessageList" aria-live="polite" role="list">
+                            { this.props.children }
+                        </ol>
+                    </div>
+                </AutoHideScrollbar>
+            );
+    }
+}

src/components/structures/auth/ForgotPassword.js

@@ -18,7 +18,7 @@ limitations under the License.
 
 import React from 'react';
 import PropTypes from 'prop-types';
-import { _t } from '../../../languageHandler';
+import { _t, _td } from '../../../languageHandler';
 import * as sdk from '../../../index';
 import Modal from "../../../Modal";
 import PasswordReset from "../../../PasswordReset";
@@ -27,7 +27,9 @@ import classNames from 'classnames';
 import AuthPage from "../../views/auth/AuthPage";
 import CountlyAnalytics from "../../../CountlyAnalytics";
 import ServerPicker from "../../views/elements/ServerPicker";
+import PassphraseField from '../../views/auth/PassphraseField';
 import {replaceableComponent} from "../../../utils/replaceableComponent";
+import { PASSWORD_MIN_SCORE } from '../../views/auth/RegistrationForm';
 
 // Phases
 // Show the forgot password inputs
@@ -137,10 +139,14 @@ export default class ForgotPassword extends React.Component {
         // refresh the server errors, just in case the server came back online
         await this._checkServerLiveliness(this.props.serverConfig);
 
+        await this['password_field'].validate({ allowEmpty: false });
+
         if (!this.state.email) {
             this.showErrorDialog(_t('The email address linked to your account must be entered.'));
         } else if (!this.state.password || !this.state.password2) {
             this.showErrorDialog(_t('A new password must be entered.'));
+        } else if (!this.state.passwordFieldValid) {
+            this.showErrorDialog(_t('Please choose a strong password'));
         } else if (this.state.password !== this.state.password2) {
             this.showErrorDialog(_t('New passwords must match each other.'));
         } else {
@@ -186,6 +192,12 @@ export default class ForgotPassword extends React.Component {
         });
     }
 
+    onPasswordValidate(result) {
+        this.setState({
+            passwordFieldValid: result.valid,
+        });
+    }
+
     renderForgot() {
         const Field = sdk.getComponent('elements.Field');
 
@@ -230,12 +242,15 @@ export default class ForgotPassword extends React.Component {
                     />
                 </div>
                 <div className="mx_AuthBody_fieldRow">
-                    <Field
+                    <PassphraseField
                         name="reset_password"
                         type="password"
-                        label={_t('New Password')}
+                        label={_td('New Password')}
                         value={this.state.password}
+                        minScore={PASSWORD_MIN_SCORE}
                         onChange={this.onInputChanged.bind(this, "password")}
+                        fieldRef={field => this['password_field'] = field}
+                        onValidate={(result) => this.onPasswordValidate(result)}
                         onFocus={() => CountlyAnalytics.instance.track("onboarding_forgot_password_newPassword_focus")}
                         onBlur={() => CountlyAnalytics.instance.track("onboarding_forgot_password_newPassword_blur")}
                         autoComplete="new-password"

src/components/views/auth/RegistrationForm.tsx

@@ -40,7 +40,7 @@ enum RegistrationField {
     PasswordConfirm = "field_password_confirm",
 }
 
-const PASSWORD_MIN_SCORE = 3; // safely unguessable: moderate protection from offline slow-hash scenario.
+export const PASSWORD_MIN_SCORE = 3; // safely unguessable: moderate protection from offline slow-hash scenario.
 
 interface IProps {
     // Values pre-filled in the input boxes when the component loads

src/components/views/dialogs/InviteDialog.tsx

@@ -29,7 +29,9 @@ import dis from "../../../dispatcher/dispatcher";
 import IdentityAuthClient from "../../../IdentityAuthClient";
 import Modal from "../../../Modal";
 import {humanizeTime} from "../../../utils/humanize";
-import createRoom, {canEncryptToAllUsers, findDMForUser, privateShouldBeEncrypted} from "../../../createRoom";
+import createRoom, {
+    canEncryptToAllUsers, ensureDMExists, findDMForUser, privateShouldBeEncrypted,
+} from "../../../createRoom";
 import {inviteMultipleToRoom, showCommunityInviteDialog} from "../../../RoomInvite";
 import {Key} from "../../../Keyboard";
 import {Action} from "../../../dispatcher/actions";
@@ -332,6 +334,7 @@ interface IInviteDialogState {
     threepidResultsMixin: { user: Member, userId: string}[];
     canUseIdentityServer: boolean;
     tryingIdentityServer: boolean;
+    consultFirst: boolean;
 
     // These two flags are used for the 'Go' button to communicate what is going on.
     busy: boolean,
@@ -380,6 +383,7 @@ export default class InviteDialog extends React.PureComponent<IInviteDialogProps
             threepidResultsMixin: [],
             canUseIdentityServer: !!MatrixClientPeg.get().getIdentityServerUrl(),
             tryingIdentityServer: false,
+            consultFirst: false,
 
             // These two flags are used for the 'Go' button to communicate what is going on.
             busy: false,
@@ -395,6 +399,10 @@ export default class InviteDialog extends React.PureComponent<IInviteDialogProps
         }
     }
 
+    private onConsultFirstChange = (ev) => {
+        this.setState({consultFirst: ev.target.checked});
+    }
+
     static buildRecents(excludedTargetIds: Set<string>): {userId: string, user: RoomMember, lastActive: number}[] {
         const rooms = DMRoomMap.shared().getUniqueRoomsWithIndividuals(); // map of userId => js-sdk Room
 
@@ -745,16 +753,34 @@ export default class InviteDialog extends React.PureComponent<IInviteDialogProps
             });
         }
 
-        this.setState({busy: true});
+        if (this.state.consultFirst) {
-        try {
+            const dmRoomId = await ensureDMExists(MatrixClientPeg.get(), targetIds[0]);
-            await this.props.call.transfer(targetIds[0]);
+
-            this.setState({busy: false});
+            dis.dispatch({
-            this.props.onFinished();
+                action: 'place_call',
-        } catch (e) {
+                type: this.props.call.type,
-            this.setState({
+                room_id: dmRoomId,
-                busy: false,
+                transferee: this.props.call,
-                errorText: _t("Failed to transfer call"),
             });
+            dis.dispatch({
+                action: 'view_room',
+                room_id: dmRoomId,
+                should_peek: false,
+                joining: false,
+            });
+            this.props.onFinished();
+        } else {
+            this.setState({busy: true});
+            try {
+                await this.props.call.transfer(targetIds[0]);
+                this.setState({busy: false});
+                this.props.onFinished();
+            } catch (e) {
+                this.setState({
+                    busy: false,
+                    errorText: _t("Failed to transfer call"),
+                });
+            }
         }
     };
 
@@ -1215,6 +1241,7 @@ export default class InviteDialog extends React.PureComponent<IInviteDialogProps
         let helpText;
         let buttonText;
         let goButtonFn;
+        let consultSection;
         let keySharingWarning = <span />;
 
         const identityServersEnabled = SettingsStore.getValue(UIFeature.IdentityServer);
@@ -1339,6 +1366,12 @@ export default class InviteDialog extends React.PureComponent<IInviteDialogProps
             title = _t("Transfer");
             buttonText = _t("Transfer");
             goButtonFn = this._transferCall;
+            consultSection = <div>
+                <label>
+                    <input type="checkbox" checked={this.state.consultFirst} onChange={this.onConsultFirstChange} />
+                    {_t("Consult first")}
+                </label>
+            </div>;
         } else {
             console.error("Unknown kind of InviteDialog: " + this.props.kind);
         }
@@ -1375,6 +1408,7 @@ export default class InviteDialog extends React.PureComponent<IInviteDialogProps
                         {this._renderSection('recents')}
                         {this._renderSection('suggestions')}
                     </div>
+                    {consultSection}
                 </div>
             </BaseDialog>
         );

src/components/views/dialogs/SeshatResetDialog.tsx

@@ -0,0 +1,54 @@
+/*
+Copyright 2021 The Matrix.org Foundation C.I.C.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+import React from 'react';
+import {_t} from "../../../languageHandler";
+import {replaceableComponent} from "../../../utils/replaceableComponent";
+
+import BaseDialog from "./BaseDialog";
+import DialogButtons from "../elements/DialogButtons";
+
+import {IDialogProps} from "./IDialogProps";
+
+@replaceableComponent("views.dialogs.SeshatResetDialog")
+export default class SeshatResetDialog extends React.PureComponent<IDialogProps> {
+    render() {
+        return (
+            <BaseDialog
+                hasCancel={true}
+                onFinished={this.props.onFinished.bind(null, false)}
+                title={_t("Reset event store?")}>
+                <div>
+                    <p>
+                        {_t("You most likely do not want to reset your event index store")}
+                        <br />
+                        {_t("If you do, please note that none of your messages will be deleted, " +
+                            "but the search experience might be degraded for a few moments" +
+                            "whilst the index is recreated",
+                        )}
+                    </p>
+                </div>
+                <DialogButtons
+                    primaryButton={_t("Reset event store")}
+                    onPrimaryButtonClick={this.props.onFinished.bind(null, true)}
+                    primaryButtonClass="danger"
+                    cancelButton={_t("Cancel")}
+                    onCancel={this.props.onFinished.bind(null, false)}
+                />
+            </BaseDialog>
+        );
+    }
+}

src/components/views/elements/EventTilePreview.tsx

@@ -19,7 +19,6 @@ import classnames from 'classnames';
 import { MatrixEvent } from 'matrix-js-sdk/src/models/event';
 
 import * as Avatar from '../../../Avatar';
-import { MatrixClientPeg } from '../../../MatrixClientPeg';
 import EventTile from '../rooms/EventTile';
 import SettingsStore from "../../../settings/SettingsStore";
 import {Layout} from "../../../settings/Layout";
@@ -41,15 +40,38 @@ interface IProps {
      * classnames to apply to the wrapper of the preview
      */
     className: string;
+
+    /**
+     * The ID of the displayed user
+     */
+    userId: string;
+
+    /**
+     * The display name of the displayed user
+     */
+    displayName?: string;
+
+    /**
+     * The mxc:// avatar URL of the displayed user
+     */
+    avatarUrl?: string;
+
+    /**
+     * Whether the EventTile should appear faded
+     */
+    faded?: boolean;
+
+    /**
+     * Callback for when the component is clicked
+     */
+    onClick?: () => void;
 }
 
-/* eslint-disable camelcase */
 interface IState {
-    userId: string;
+    message: string;
-    displayname: string;
+    faded: boolean;
-    avatar_url: string;
+    eventTileKey: number;
 }
-/* eslint-enable camelcase */
 
 const AVATAR_SIZE = 32;
 
@@ -57,45 +79,42 @@ const AVATAR_SIZE = 32;
 export default class EventTilePreview extends React.Component<IProps, IState> {
     constructor(props: IProps) {
         super(props);
-
         this.state = {
-            userId: "@erim:fink.fink",
+            message: props.message,
-            displayname: "Erimayas Fink",
+            faded: !!props.faded,
-            avatar_url: null,
+            eventTileKey: 0,
         };
     }
 
-    async componentDidMount() {
+    changeMessage(message: string) {
-        // Fetch current user data
-        const client = MatrixClientPeg.get();
-        const userId = client.getUserId();
-        const profileInfo = await client.getProfileInfo(userId);
-        const avatarUrl = profileInfo.avatar_url;
-
         this.setState({
-            userId,
+            message,
-            displayname: profileInfo.displayname,
+            // Change the EventTile key to force React to create a new instance
-            avatar_url: avatarUrl,
+            eventTileKey: this.state.eventTileKey + 1,
         });
     }
 
-    private fakeEvent({userId, displayname, avatar_url: avatarUrl}: IState) {
+    unfade() {
+        this.setState({ faded: false });
+    }
+
+    private fakeEvent({message}: IState) {
         // Fake it till we make it
         /* eslint-disable quote-props */
         const rawEvent = {
             type: "m.room.message",
-            sender: userId,
+            sender: this.props.userId,
             content: {
                 "m.new_content": {
                     msgtype: "m.text",
-                    body: this.props.message,
+                    body: message,
-                    displayname: displayname,
+                    displayname: this.props.displayName,
-                    avatar_url: avatarUrl,
+                    avatar_url: this.props.avatarUrl,
                 },
                 msgtype: "m.text",
-                body: this.props.message,
+                body: message,
-                displayname: displayname,
+                displayname: this.props.displayName,
-                avatar_url: avatarUrl,
+                avatar_url: this.props.avatarUrl,
             },
             unsigned: {
                 age: 97,
@@ -108,12 +127,15 @@ export default class EventTilePreview extends React.Component<IProps, IState> {
 
         // Fake it more
         event.sender = {
-            name: displayname,
+            name: this.props.displayName,
-            userId: userId,
+            userId: this.props.userId,
             getAvatarUrl: (..._) => {
-                return Avatar.avatarUrlForUser({avatarUrl}, AVATAR_SIZE, AVATAR_SIZE, "crop");
+                return Avatar.avatarUrlForUser(
+                    { avatarUrl: this.props.avatarUrl },
+                    AVATAR_SIZE, AVATAR_SIZE, "crop",
+                );
             },
-            getMxcAvatarUrl: () => avatarUrl,
+            getMxcAvatarUrl: () => this.props.avatarUrl,
         };
 
         return event;
@@ -125,10 +147,12 @@ export default class EventTilePreview extends React.Component<IProps, IState> {
         const className = classnames(this.props.className, {
             "mx_IRCLayout": this.props.layout == Layout.IRC,
             "mx_GroupLayout": this.props.layout == Layout.Group,
+            "mx_EventTilePreview_faded": this.state.faded,
         });
 
-        return <div className={className}>
+        return <div className={className} onClick={this.props.onClick}>
             <EventTile
+                key={this.state.eventTileKey}
                 mxEvent={event}
                 layout={this.props.layout}
                 enableFlair={SettingsStore.getValue(UIFeature.Flair)}

src/components/views/rooms/BasicMessageComposer.tsx

@@ -485,16 +485,14 @@ export default class BasicMessageEditor extends React.Component<IProps, IState>
         if (model.autoComplete && model.autoComplete.hasCompletions()) {
             const autoComplete = model.autoComplete;
             switch (autocompleteAction) {
+                case AutocompleteAction.CompleteOrPrevSelection:
                 case AutocompleteAction.PrevSelection:
-                    autoComplete.onUpArrow(event);
+                    autoComplete.selectPreviousSelection();
                     handled = true;
                     break;
+                case AutocompleteAction.CompleteOrNextSelection:
                 case AutocompleteAction.NextSelection:
-                    autoComplete.onDownArrow(event);
+                    autoComplete.selectNextSelection();
-                    handled = true;
-                    break;
-                case AutocompleteAction.ApplySelection:
-                    autoComplete.onTab(event);
                     handled = true;
                     break;
                 case AutocompleteAction.Cancel:
@@ -504,8 +502,10 @@ export default class BasicMessageEditor extends React.Component<IProps, IState>
                 default:
                     return; // don't preventDefault on anything else
             }
-        } else if (autocompleteAction === AutocompleteAction.ApplySelection) {
+        } else if (autocompleteAction === AutocompleteAction.CompleteOrPrevSelection
-            this.tabCompleteName(event);
+            || autocompleteAction === AutocompleteAction.CompleteOrNextSelection) {
+            // there is no current autocomplete window, try to open it
+            this.tabCompleteName();
             handled = true;
         } else if (event.key === Key.BACKSPACE || event.key === Key.DELETE) {
             this.formatBarRef.current.hide();
@@ -517,7 +517,7 @@ export default class BasicMessageEditor extends React.Component<IProps, IState>
         }
     };
 
-    private async tabCompleteName(event: React.KeyboardEvent) {
+    private async tabCompleteName() {
         try {
             await new Promise<void>(resolve => this.setState({showVisualBell: false}, resolve));
             const {model} = this.props;
@@ -540,7 +540,7 @@ export default class BasicMessageEditor extends React.Component<IProps, IState>
 
             // Don't try to do things with the autocomplete if there is none shown
             if (model.autoComplete) {
-                await model.autoComplete.onTab(event);
+                await model.autoComplete.startSelection();
                 if (!model.autoComplete.hasSelection()) {
                     this.setState({showVisualBell: true});
                     model.autoComplete.close();

src/components/views/rooms/EventTile.js

@@ -936,7 +936,7 @@ export default class EventTile extends React.Component {
             );
 
         const TooltipButton = sdk.getComponent('elements.TooltipButton');
-        const keyRequestInfo = isEncryptionFailure ?
+        const keyRequestInfo = isEncryptionFailure && !isRedacted ?
             <div className="mx_EventTile_keyRequestInfo">
                 <span className="mx_EventTile_keyRequestInfo_text">
                     { keyRequestInfoContent }

src/components/views/rooms/RoomPreviewBar.js

@@ -25,6 +25,7 @@ import classNames from 'classnames';
 import { _t } from '../../../languageHandler';
 import SdkConfig from "../../../SdkConfig";
 import IdentityAuthClient from '../../../IdentityAuthClient';
+import SettingsStore from "../../../settings/SettingsStore";
 import {CommunityPrototypeStore} from "../../../stores/CommunityPrototypeStore";
 import {UPDATE_EVENT} from "../../../stores/AsyncStore";
 import {replaceableComponent} from "../../../utils/replaceableComponent";
@@ -302,10 +303,12 @@ export default class RoomPreviewBar extends React.Component {
         const brand = SdkConfig.get().brand;
         const Spinner = sdk.getComponent('elements.Spinner');
         const AccessibleButton = sdk.getComponent('elements.AccessibleButton');
+        const EventTilePreview = sdk.getComponent('elements.EventTilePreview');
 
         let showSpinner = false;
         let title;
         let subTitle;
+        let reasonElement;
         let primaryActionHandler;
         let primaryActionLabel;
         let secondaryActionHandler;
@@ -491,6 +494,29 @@ export default class RoomPreviewBar extends React.Component {
                     primaryActionLabel = _t("Accept");
                 }
 
+                const myUserId = MatrixClientPeg.get().getUserId();
+                const reason = this.props.room.currentState.getMember(myUserId).events.member.event.content.reason;
+                if (reason) {
+                    this.reasonElement = React.createRef();
+                    // We hide the reason for invitation by default, since it can be a
+                    // vector for spam/harassment.
+                    const showReason = () => {
+                        this.reasonElement.current.unfade();
+                        this.reasonElement.current.changeMessage(reason);
+                    };
+                    reasonElement = <EventTilePreview
+                        ref={this.reasonElement}
+                        onClick={showReason}
+                        className="mx_RoomPreviewBar_reason"
+                        message={_t("Invite messages are hidden by default. Click to show the message.")}
+                        layout={SettingsStore.getValue("layout")}
+                        userId={inviteMember.userId}
+                        displayName={inviteMember.rawDisplayName}
+                        avatarUrl={inviteMember.events.member.event.content.avatar_url}
+                        faded={true}
+                    />;
+                }
+
                 primaryActionHandler = this.props.onJoinClick;
                 secondaryActionLabel = _t("Reject");
                 secondaryActionHandler = this.props.onRejectClick;
@@ -582,6 +608,7 @@ export default class RoomPreviewBar extends React.Component {
                     { titleElement }
                     { subTitleElements }
                 </div>
+                { reasonElement }
                 <div className="mx_RoomPreviewBar_actions">
                     { secondaryButton }
                     { extraComponents }

src/components/views/settings/ChangePassword.js

@@ -28,13 +28,12 @@ import Modal from "../../../Modal";
 import PassphraseField from "../auth/PassphraseField";
 import CountlyAnalytics from "../../../CountlyAnalytics";
 import {replaceableComponent} from "../../../utils/replaceableComponent";
+import { PASSWORD_MIN_SCORE } from '../auth/RegistrationForm';
 
 const FIELD_OLD_PASSWORD = 'field_old_password';
 const FIELD_NEW_PASSWORD = 'field_new_password';
 const FIELD_NEW_PASSWORD_CONFIRM = 'field_new_password_confirm';
 
-const PASSWORD_MIN_SCORE = 3; // safely unguessable: moderate protection from offline slow-hash scenario.
-
 @replaceableComponent("views.settings.ChangePassword")
 export default class ChangePassword extends React.Component {
     static propTypes = {

src/components/views/settings/EventIndexPanel.js

@@ -26,6 +26,7 @@ import {formatBytes, formatCountLong} from "../../../utils/FormattingUtils";
 import EventIndexPeg from "../../../indexing/EventIndexPeg";
 import {SettingLevel} from "../../../settings/SettingLevel";
 import {replaceableComponent} from "../../../utils/replaceableComponent";
+import SeshatResetDialog from '../dialogs/SeshatResetDialog';
 
 @replaceableComponent("views.settings.EventIndexPanel")
 export default class EventIndexPanel extends React.Component {
@@ -122,6 +123,20 @@ export default class EventIndexPanel extends React.Component {
         await this.updateState();
     }
 
+    _confirmEventStoreReset = () => {
+        const self = this;
+        const { close } = Modal.createDialog(SeshatResetDialog, {
+            onFinished: async (success) => {
+                if (success) {
+                    await SettingsStore.setValue('enableEventIndexing', null, SettingLevel.DEVICE, false);
+                    await EventIndexPeg.deleteEventIndex();
+                    await self._onEnable();
+                    close();
+                }
+            },
+        });
+    }
+
     render() {
         let eventIndexingSettings = null;
         const InlineSpinner = sdk.getComponent('elements.InlineSpinner');
@@ -167,7 +182,7 @@ export default class EventIndexPanel extends React.Component {
             );
         } else if (EventIndexPeg.platformHasSupport() && !EventIndexPeg.supportIsInstalled()) {
             const nativeLink = (
-                "https://github.com/vector-im/element-web/blob/develop/" +
+                "https://github.com/vector-im/element-desktop/blob/develop/" +
                 "docs/native-node-modules.md#" +
                 "adding-seshat-for-search-in-e2e-encrypted-rooms"
             );
@@ -212,7 +227,10 @@ export default class EventIndexPanel extends React.Component {
             eventIndexingSettings = (
                 <div className='mx_SettingsTab_subsectionText'>
                     <p>
-                        {_t("Message search initialisation failed")}
+                        {this.state.enabling
+                            ? <InlineSpinner />
+                            : _t("Message search initilisation failed")
+                        }
                     </p>
                     {EventIndexPeg.error && (
                     <details>
@@ -220,6 +238,11 @@ export default class EventIndexPanel extends React.Component {
                         <code>
                             {EventIndexPeg.error.message}
                         </code>
+                        <p>
+                            <AccessibleButton key="delete" kind="danger" onClick={this._confirmEventStoreReset}>
+                                {_t("Reset")}
+                            </AccessibleButton>
+                        </p>
                     </details>
                     )}
 

src/components/views/settings/tabs/user/AppearanceUserSettingsTab.tsx

@@ -18,6 +18,7 @@ limitations under the License.
 import React from 'react';
 import {_t} from "../../../../../languageHandler";
 import SdkConfig from "../../../../../SdkConfig";
+import { MatrixClientPeg } from '../../../../../MatrixClientPeg';
 import SettingsStore from "../../../../../settings/SettingsStore";
 import { enumerateThemes } from "../../../../../theme";
 import ThemeWatcher from "../../../../../settings/watchers/ThemeWatcher";
@@ -63,6 +64,10 @@ interface IState extends IThemeState {
     systemFont: string;
     showAdvanced: boolean;
     layout: Layout;
+    // User profile data for the message preview
+    userId: string;
+    displayName: string;
+    avatarUrl: string;
 }
 
 @replaceableComponent("views.settings.tabs.user.AppearanceUserSettingsTab")
@@ -84,9 +89,25 @@ export default class AppearanceUserSettingsTab extends React.Component<IProps, I
             systemFont: SettingsStore.getValue("systemFont"),
             showAdvanced: false,
             layout: SettingsStore.getValue("layout"),
+            userId: "@erim:fink.fink",
+            displayName: "Erimayas Fink",
+            avatarUrl: null,
         };
     }
 
+    async componentDidMount() {
+        // Fetch the current user profile for the message preview
+        const client = MatrixClientPeg.get();
+        const userId = client.getUserId();
+        const profileInfo = await client.getProfileInfo(userId);
+
+        this.setState({
+            userId,
+            displayName: profileInfo.displayname,
+            avatarUrl: profileInfo.avatar_url,
+        });
+    }
+
     private calculateThemeState(): IThemeState {
         // We have to mirror the logic from ThemeWatcher.getEffectiveTheme so we
         // show the right values for things.
@@ -307,6 +328,9 @@ export default class AppearanceUserSettingsTab extends React.Component<IProps, I
                 className="mx_AppearanceUserSettingsTab_fontSlider_preview"
                 message={this.MESSAGE_PREVIEW_TEXT}
                 layout={this.state.layout}
+                userId={this.state.userId}
+                displayName={this.state.displayName}
+                avatarUrl={this.state.avatarUrl}
             />
             <div className="mx_AppearanceUserSettingsTab_fontSlider">
                 <div className="mx_AppearanceUserSettingsTab_fontSlider_smallText">Aa</div>

src/components/views/voip/CallView.tsx

@@ -364,6 +364,11 @@ export default class CallView extends React.Component<IProps, IState> {
         CallHandler.sharedInstance().setActiveCallRoomId(userFacingRoomId);
     }
 
+    private onTransferClick = () => {
+        const transfereeCall = CallHandler.sharedInstance().getTransfereeForCallId(this.props.call.callId);
+        this.props.call.transferToCall(transfereeCall);
+    }
+
     public render() {
         const client = MatrixClientPeg.get();
         const callRoomId = CallHandler.roomIdForCall(this.props.call);
@@ -479,25 +484,52 @@ export default class CallView extends React.Component<IProps, IState> {
         // for voice calls (fills the bg)
         let contentView: React.ReactNode;
 
+        const transfereeCall = CallHandler.sharedInstance().getTransfereeForCallId(this.props.call.callId);
         const isOnHold = this.state.isLocalOnHold || this.state.isRemoteOnHold;
-        let onHoldText = null;
+        let holdTransferContent;
-        if (this.state.isRemoteOnHold) {
+        if (transfereeCall) {
-            const holdString = CallHandler.sharedInstance().hasAnyUnheldCall() ?
+            const transferTargetRoom = MatrixClientPeg.get().getRoom(CallHandler.roomIdForCall(this.props.call));
-                _td("You held the call <a>Switch</a>") : _td("You held the call <a>Resume</a>");
+            const transferTargetName = transferTargetRoom ? transferTargetRoom.name : _t("unknown person");
-            onHoldText = _t(holdString, {}, {
+
-                a: sub => <AccessibleButton kind="link" onClick={this.onCallResumeClick}>
+            const transfereeRoom = MatrixClientPeg.get().getRoom(
-                    {sub}
+                CallHandler.roomIdForCall(transfereeCall),
-                </AccessibleButton>,
+            );
-            });
+            const transfereeName = transfereeRoom ? transfereeRoom.name : _t("unknown person");
-        } else if (this.state.isLocalOnHold) {
+
-            onHoldText = _t("%(peerName)s held the call", {
+            holdTransferContent = <div className="mx_CallView_holdTransferContent">
-                peerName: this.props.call.getOpponentMember().name,
+                {_t(
-            });
+                    "Consulting with %(transferTarget)s. <a>Transfer to %(transferee)s</a>",
+                    {
+                        transferTarget: transferTargetName,
+                        transferee: transfereeName,
+                    },
+                    {
+                        a: sub => <AccessibleButton kind="link" onClick={this.onTransferClick}>{sub}</AccessibleButton>,
+                    },
+                )}
+            </div>;
+        } else if (isOnHold) {
+            let onHoldText = null;
+            if (this.state.isRemoteOnHold) {
+                const holdString = CallHandler.sharedInstance().hasAnyUnheldCall() ?
+                    _td("You held the call <a>Switch</a>") : _td("You held the call <a>Resume</a>");
+                onHoldText = _t(holdString, {}, {
+                    a: sub => <AccessibleButton kind="link" onClick={this.onCallResumeClick}>
+                        {sub}
+                    </AccessibleButton>,
+                });
+            } else if (this.state.isLocalOnHold) {
+                onHoldText = _t("%(peerName)s held the call", {
+                    peerName: this.props.call.getOpponentMember().name,
+                });
+            }
+            holdTransferContent = <div className="mx_CallView_holdTransferContent">
+                {onHoldText}
+            </div>;
         }
 
         if (this.props.call.type === CallType.Video) {
             let localVideoFeed = null;
-            let onHoldContent = null;
             let onHoldBackground = null;
             const backgroundStyle: CSSProperties = {};
             const containerClasses = classNames({
@@ -505,9 +537,6 @@ export default class CallView extends React.Component<IProps, IState> {
                 mx_CallView_video_hold: isOnHold,
             });
             if (isOnHold) {
-                onHoldContent = <div className="mx_CallView_video_holdContent">
-                    {onHoldText}
-                </div>;
                 const backgroundAvatarUrl = avatarUrlForMember(
                     // is it worth getting the size of the div to pass here?
                     this.props.call.getOpponentMember(), 1024, 1024, 'crop',
@@ -534,7 +563,7 @@ export default class CallView extends React.Component<IProps, IState> {
                     maxHeight={maxVideoHeight}
                 />
                 {localVideoFeed}
-                {onHoldContent}
+                {holdTransferContent}
                 {callControls}
             </div>;
         } else {
@@ -554,7 +583,7 @@ export default class CallView extends React.Component<IProps, IState> {
                         />
                     </div>
                 </div>
-                <div className="mx_CallView_voice_holdText">{onHoldText}</div>
+                {holdTransferContent}
                 {callControls}
             </div>;
         }

src/editor/autocomplete.ts

@@ -68,24 +68,24 @@ export default class AutocompleteWrapperModel {
         this.updateCallback({close: true});
     }
 
-    public async onTab(e: KeyboardEvent) {
+    /**
+     * If there is no current autocompletion, start one and move to the first selection.
+     */
+    public async startSelection() {
         const acComponent = this.getAutocompleterComponent();
-
         if (acComponent.countCompletions() === 0) {
             // Force completions to show for the text currently entered
             await acComponent.forceComplete();
             // Select the first item by moving "down"
             await acComponent.moveSelection(+1);
-        } else {
-            await acComponent.moveSelection(e.shiftKey ? -1 : +1);
         }
     }
 
-    public onUpArrow(e: KeyboardEvent) {
+    public selectPreviousSelection() {
         this.getAutocompleterComponent().moveSelection(-1);
     }
 
-    public onDownArrow(e: KeyboardEvent) {
+    public selectNextSelection() {
         this.getAutocompleterComponent().moveSelection(+1);
     }
 

src/i18n/strings/en_EN.json

@@ -881,6 +881,8 @@
     "sends fireworks": "sends fireworks",
     "Sends the given message with snowfall": "Sends the given message with snowfall",
     "sends snowfall": "sends snowfall",
+    "unknown person": "unknown person",
+    "Consulting with %(transferTarget)s. <a>Transfer to %(transferee)s</a>": "Consulting with %(transferTarget)s. <a>Transfer to %(transferee)s</a>",
     "You held the call <a>Switch</a>": "You held the call <a>Switch</a>",
     "You held the call <a>Resume</a>": "You held the call <a>Resume</a>",
     "%(peerName)s held the call": "%(peerName)s held the call",
@@ -1084,7 +1086,7 @@
     "Securely cache encrypted messages locally for them to appear in search results.": "Securely cache encrypted messages locally for them to appear in search results.",
     "%(brand)s is missing some components required for securely caching encrypted messages locally. If you'd like to experiment with this feature, build a custom %(brand)s Desktop with <nativeLink>search components added</nativeLink>.": "%(brand)s is missing some components required for securely caching encrypted messages locally. If you'd like to experiment with this feature, build a custom %(brand)s Desktop with <nativeLink>search components added</nativeLink>.",
     "%(brand)s can't securely cache encrypted messages locally while running in a web browser. Use <desktopLink>%(brand)s Desktop</desktopLink> for encrypted messages to appear in search results.": "%(brand)s can't securely cache encrypted messages locally while running in a web browser. Use <desktopLink>%(brand)s Desktop</desktopLink> for encrypted messages to appear in search results.",
-    "Message search initialisation failed": "Message search initialisation failed",
+    "Message search initilisation failed": "Message search initilisation failed",
     "Connecting to integration manager...": "Connecting to integration manager...",
     "Cannot connect to integration manager": "Cannot connect to integration manager",
     "The integration manager is offline or it cannot reach your homeserver.": "The integration manager is offline or it cannot reach your homeserver.",
@@ -1576,6 +1578,7 @@
     "Start chatting": "Start chatting",
     "Do you want to join %(roomName)s?": "Do you want to join %(roomName)s?",
     "<userName/> invited you": "<userName/> invited you",
+    "Invite messages are hidden by default. Click to show the message.": "Invite messages are hidden by default. Click to show the message.",
     "Reject": "Reject",
     "Reject & Ignore user": "Reject & Ignore user",
     "You're previewing %(roomName)s. Want to join it?": "You're previewing %(roomName)s. Want to join it?",
@@ -2215,6 +2218,7 @@
     "Invite someone using their name, username (like <userId/>) or <a>share this room</a>.": "Invite someone using their name, username (like <userId/>) or <a>share this room</a>.",
     "Invited people will be able to read old messages.": "Invited people will be able to read old messages.",
     "Transfer": "Transfer",
+    "Consult first": "Consult first",
     "a new master key signature": "a new master key signature",
     "a new cross-signing key signature": "a new cross-signing key signature",
     "a device cross-signing signature": "a device cross-signing signature",
@@ -2305,6 +2309,10 @@
     "Use your preferred Matrix homeserver if you have one, or host your own.": "Use your preferred Matrix homeserver if you have one, or host your own.",
     "Learn more": "Learn more",
     "About homeservers": "About homeservers",
+    "Reset event store?": "Reset event store?",
+    "You most likely do not want to reset your event index store": "You most likely do not want to reset your event index store",
+    "If you do, please note that none of your messages will be deleted, but the search experience might be degraded for a few momentswhilst the index is recreated": "If you do, please note that none of your messages will be deleted, but the search experience might be degraded for a few momentswhilst the index is recreated",
+    "Reset event store": "Reset event store",
     "Sign out and remove encryption keys?": "Sign out and remove encryption keys?",
     "Clear Storage and Sign Out": "Clear Storage and Sign Out",
     "Send Logs": "Send Logs",
@@ -2693,6 +2701,7 @@
     "Failed to send email": "Failed to send email",
     "The email address linked to your account must be entered.": "The email address linked to your account must be entered.",
     "A new password must be entered.": "A new password must be entered.",
+    "Please choose a strong password": "Please choose a strong password",
     "New passwords must match each other.": "New passwords must match each other.",
     "Changing your password will reset any end-to-end encryption keys on all of your sessions, making encrypted chat history unreadable. Set up Key Backup or export your room keys from another session before resetting your password.": "Changing your password will reset any end-to-end encryption keys on all of your sessions, making encrypted chat history unreadable. Set up Key Backup or export your room keys from another session before resetting your password.",
     "New Password": "New Password",

src/stores/room-list/ListLayout.ts

@@ -82,7 +82,7 @@ export class ListLayout {
 
     public get defaultVisibleTiles(): number {
         // This number is what "feels right", and mostly subject to design's opinion.
-        return 5;
+        return 8;
     }
 
     public tilesWithPadding(n: number, paddingPx: number): number {

src/stores/room-list/RoomListStore.ts

@@ -1,6 +1,5 @@
 /*
-Copyright 2018, 2019 New Vector Ltd
+Copyright 2018-2021 The Matrix.org Foundation C.I.C.
-Copyright 2020 The Matrix.org Foundation C.I.C.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -15,27 +14,27 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-import { MatrixClient } from "matrix-js-sdk/src/client";
+import {MatrixClient} from "matrix-js-sdk/src/client";
 import SettingsStore from "../../settings/SettingsStore";
-import { DefaultTagID, isCustomTag, OrderedDefaultTagIDs, RoomUpdateCause, TagID } from "./models";
+import {DefaultTagID, isCustomTag, OrderedDefaultTagIDs, RoomUpdateCause, TagID} from "./models";
-import { Room } from "matrix-js-sdk/src/models/room";
+import {Room} from "matrix-js-sdk/src/models/room";
-import { IListOrderingMap, ITagMap, ITagSortingMap, ListAlgorithm, SortAlgorithm } from "./algorithms/models";
+import {IListOrderingMap, ITagMap, ITagSortingMap, ListAlgorithm, SortAlgorithm} from "./algorithms/models";
-import { ActionPayload } from "../../dispatcher/payloads";
+import {ActionPayload} from "../../dispatcher/payloads";
 import defaultDispatcher from "../../dispatcher/dispatcher";
-import { readReceiptChangeIsFor } from "../../utils/read-receipts";
+import {readReceiptChangeIsFor} from "../../utils/read-receipts";
-import { FILTER_CHANGED, IFilterCondition } from "./filters/IFilterCondition";
+import {FILTER_CHANGED, FilterKind, IFilterCondition} from "./filters/IFilterCondition";
-import { TagWatcher } from "./TagWatcher";
+import {TagWatcher} from "./TagWatcher";
 import RoomViewStore from "../RoomViewStore";
-import { Algorithm, LIST_UPDATED_EVENT } from "./algorithms/Algorithm";
+import {Algorithm, LIST_UPDATED_EVENT} from "./algorithms/Algorithm";
-import { EffectiveMembership, getEffectiveMembership } from "../../utils/membership";
+import {EffectiveMembership, getEffectiveMembership} from "../../utils/membership";
-import { isNullOrUndefined } from "matrix-js-sdk/src/utils";
+import {isNullOrUndefined} from "matrix-js-sdk/src/utils";
 import RoomListLayoutStore from "./RoomListLayoutStore";
-import { MarkedExecution } from "../../utils/MarkedExecution";
+import {MarkedExecution} from "../../utils/MarkedExecution";
-import { AsyncStoreWithClient } from "../AsyncStoreWithClient";
+import {AsyncStoreWithClient} from "../AsyncStoreWithClient";
-import { NameFilterCondition } from "./filters/NameFilterCondition";
+import {NameFilterCondition} from "./filters/NameFilterCondition";
-import { RoomNotificationStateStore } from "../notifications/RoomNotificationStateStore";
+import {RoomNotificationStateStore} from "../notifications/RoomNotificationStateStore";
-import { VisibilityProvider } from "./filters/VisibilityProvider";
+import {VisibilityProvider} from "./filters/VisibilityProvider";
-import { SpaceWatcher } from "./SpaceWatcher";
+import {SpaceWatcher} from "./SpaceWatcher";
 
 interface IState {
     tagsEnabled?: boolean;
@@ -57,6 +56,7 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
     private initialListsGenerated = false;
     private algorithm = new Algorithm();
     private filterConditions: IFilterCondition[] = [];
+    private prefilterConditions: IFilterCondition[] = [];
     private tagWatcher: TagWatcher;
     private spaceWatcher: SpaceWatcher;
     private updateFn = new MarkedExecution(() => {
@@ -104,6 +104,7 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
     public async resetStore() {
         await this.reset();
         this.filterConditions = [];
+        this.prefilterConditions = [];
         this.initialListsGenerated = false;
         this.setupWatchers();
 
@@ -435,6 +436,39 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
         }
     }
 
+    private async recalculatePrefiltering() {
+        if (!this.algorithm) return;
+        if (!this.algorithm.hasTagSortingMap) return; // we're still loading
+
+        if (SettingsStore.getValue("advancedRoomListLogging")) {
+            // TODO: Remove debug: https://github.com/vector-im/element-web/issues/14602
+            console.log("Calculating new prefiltered room list");
+        }
+
+        // Inhibit updates because we're about to lie heavily to the algorithm
+        this.algorithm.updatesInhibited = true;
+
+        // Figure out which rooms are about to be valid, and the state of affairs
+        const rooms = this.getPlausibleRooms();
+        const currentSticky = this.algorithm.stickyRoom;
+        const stickyIsStillPresent = currentSticky && rooms.includes(currentSticky);
+
+        // Reset the sticky room before resetting the known rooms so the algorithm
+        // doesn't freak out.
+        await this.algorithm.setStickyRoom(null);
+        await this.algorithm.setKnownRooms(rooms);
+
+        // Set the sticky room back, if needed, now that we have updated the store.
+        // This will use relative stickyness to the new room set.
+        if (stickyIsStillPresent) {
+            await this.algorithm.setStickyRoom(currentSticky);
+        }
+
+        // Finally, mark an update and resume updates from the algorithm
+        this.updateFn.mark();
+        this.algorithm.updatesInhibited = false;
+    }
+
     public async setTagSorting(tagId: TagID, sort: SortAlgorithm) {
         await this.setAndPersistTagSorting(tagId, sort);
         this.updateFn.trigger();
@@ -557,6 +591,34 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
         this.updateFn.trigger();
     };
 
+    private onPrefilterUpdated = async () => {
+        await this.recalculatePrefiltering();
+        this.updateFn.trigger();
+    };
+
+    private getPlausibleRooms(): Room[] {
+        if (!this.matrixClient) return [];
+
+        let rooms = [
+            ...this.matrixClient.getVisibleRooms(),
+            // also show space invites in the room list
+            ...this.matrixClient.getRooms().filter(r => r.isSpaceRoom() && r.getMyMembership() === "invite"),
+        ].filter(r => VisibilityProvider.instance.isRoomVisible(r));
+
+        if (this.prefilterConditions.length > 0) {
+            rooms = rooms.filter(r => {
+                for (const filter of this.prefilterConditions) {
+                    if (!filter.isVisible(r)) {
+                        return false;
+                    }
+                }
+                return true;
+            });
+        }
+
+        return rooms;
+    }
+
     /**
      * Regenerates the room whole room list, discarding any previous results.
      *
@@ -568,11 +630,7 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
     public async regenerateAllLists({trigger = true}) {
         console.warn("Regenerating all room lists");
 
-        const rooms = [
+        const rooms = this.getPlausibleRooms();
-            ...this.matrixClient.getVisibleRooms(),
-            // also show space invites in the room list
-            ...this.matrixClient.getRooms().filter(r => r.isSpaceRoom() && r.getMyMembership() === "invite"),
-        ].filter(r => VisibilityProvider.instance.isRoomVisible(r));
 
         const customTags = new Set<TagID>();
         if (this.state.tagsEnabled) {
@@ -601,24 +659,44 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
         if (trigger) this.updateFn.trigger();
     }
 
+    /**
+     * Adds a filter condition to the room list store. Filters may be applied async,
+     * and thus might not cause an update to the store immediately.
+     * @param {IFilterCondition} filter The filter condition to add.
+     */
     public addFilter(filter: IFilterCondition): void {
         if (SettingsStore.getValue("advancedRoomListLogging")) {
             // TODO: Remove debug: https://github.com/vector-im/element-web/issues/14602
             console.log("Adding filter condition:", filter);
         }
-        this.filterConditions.push(filter);
+        let promise = Promise.resolve();
-        if (this.algorithm) {
+        if (filter.kind === FilterKind.Prefilter) {
-            this.algorithm.addFilterCondition(filter);
+            filter.on(FILTER_CHANGED, this.onPrefilterUpdated);
+            this.prefilterConditions.push(filter);
+            promise = this.recalculatePrefiltering();
+        } else {
+            this.filterConditions.push(filter);
+            if (this.algorithm) {
+                this.algorithm.addFilterCondition(filter);
+            }
         }
-        this.updateFn.trigger();
+        promise.then(() => this.updateFn.trigger());
     }
 
+    /**
+     * Removes a filter condition from the room list store. If the filter was
+     * not previously added to the room list store, this will no-op. The effects
+     * of removing a filter may be applied async and therefore might not cause
+     * an update right away.
+     * @param {IFilterCondition} filter The filter condition to remove.
+     */
     public removeFilter(filter: IFilterCondition): void {
         if (SettingsStore.getValue("advancedRoomListLogging")) {
             // TODO: Remove debug: https://github.com/vector-im/element-web/issues/14602
             console.log("Removing filter condition:", filter);
         }
-        const idx = this.filterConditions.indexOf(filter);
+        let promise = Promise.resolve();
+        let idx = this.filterConditions.indexOf(filter);
         if (idx >= 0) {
             this.filterConditions.splice(idx, 1);
 
@@ -626,7 +704,13 @@ export class RoomListStoreClass extends AsyncStoreWithClient<IState> {
                 this.algorithm.removeFilterCondition(filter);
             }
         }
-        this.updateFn.trigger();
+        idx = this.prefilterConditions.indexOf(filter);
+        if (idx >= 0) {
+            filter.off(FILTER_CHANGED, this.onPrefilterUpdated);
+            this.prefilterConditions.splice(idx, 1);
+            promise = this.recalculatePrefiltering();
+        }
+        promise.then(() => this.updateFn.trigger());
     }
 
     /**

src/stores/room-list/algorithms/Algorithm.ts

@@ -1,5 +1,5 @@
 /*
-Copyright 2020 The Matrix.org Foundation C.I.C.
+Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -18,8 +18,7 @@ import { Room } from "matrix-js-sdk/src/models/room";
 import { isNullOrUndefined } from "matrix-js-sdk/src/utils";
 import DMRoomMap from "../../../utils/DMRoomMap";
 import { EventEmitter } from "events";
-import { arrayDiff, arrayHasDiff, ArrayUtil } from "../../../utils/arrays";
+import { arrayDiff, arrayHasDiff } from "../../../utils/arrays";
-import { getEnumValues } from "../../../utils/enums";
 import { DefaultTagID, RoomUpdateCause, TagID } from "../models";
 import {
     IListOrderingMap,
@@ -29,7 +28,7 @@ import {
     ListAlgorithm,
     SortAlgorithm,
 } from "./models";
-import { FILTER_CHANGED, FilterPriority, IFilterCondition } from "../filters/IFilterCondition";
+import { FILTER_CHANGED, IFilterCondition } from "../filters/IFilterCondition";
 import { EffectiveMembership, getEffectiveMembership, splitRoomsByMembership } from "../../../utils/membership";
 import { OrderingAlgorithm } from "./list-ordering/OrderingAlgorithm";
 import { getListAlgorithmInstance } from "./list-ordering";
@@ -79,6 +78,11 @@ export class Algorithm extends EventEmitter {
     private allowedByFilter: Map<IFilterCondition, Room[]> = new Map<IFilterCondition, Room[]>();
     private allowedRoomsByFilters: Set<Room> = new Set<Room>();
 
+    /**
+     * Set to true to suspend emissions of algorithm updates.
+     */
+    public updatesInhibited = false;
+
     public constructor() {
         super();
     }
@@ -87,6 +91,14 @@ export class Algorithm extends EventEmitter {
         return this._stickyRoom ? this._stickyRoom.room : null;
     }
 
+    public get knownRooms(): Room[] {
+        return this.rooms;
+    }
+
+    public get hasTagSortingMap(): boolean {
+        return !!this.sortAlgorithms;
+    }
+
     protected get hasFilters(): boolean {
         return this.allowedByFilter.size > 0;
     }
@@ -164,7 +176,7 @@ export class Algorithm extends EventEmitter {
 
             // If we removed the last filter, tell consumers that we've "updated" our filtered
             // view. This will trick them into getting the complete room list.
-            if (!this.hasFilters) {
+            if (!this.hasFilters && !this.updatesInhibited) {
                 this.emit(LIST_UPDATED_EVENT);
             }
         }
@@ -174,6 +186,7 @@ export class Algorithm extends EventEmitter {
         await this.recalculateFilteredRooms();
 
         // re-emit the update so the list store can fire an off-cycle update if needed
+        if (this.updatesInhibited) return;
         this.emit(FILTER_CHANGED);
     }
 
@@ -299,6 +312,7 @@ export class Algorithm extends EventEmitter {
         this.recalculateStickyRoom();
 
         // Finally, trigger an update
+        if (this.updatesInhibited) return;
         this.emit(LIST_UPDATED_EVENT);
     }
 
@@ -309,10 +323,6 @@ export class Algorithm extends EventEmitter {
 
         console.warn("Recalculating filtered room list");
         const filters = Array.from(this.allowedByFilter.keys());
-        const orderedFilters = new ArrayUtil(filters)
-            .groupBy(f => f.relativePriority)
-            .orderBy(getEnumValues(FilterPriority))
-            .value;
         const newMap: ITagMap = {};
         for (const tagId of Object.keys(this.cachedRooms)) {
             // Cheaply clone the rooms so we can more easily do operations on the list.
@@ -320,18 +330,9 @@ export class Algorithm extends EventEmitter {
             // to the rooms we know will be deduped by the Set.
             const rooms = this.cachedRooms[tagId].map(r => r); // cheap clone
             this.tryInsertStickyRoomToFilterSet(rooms, tagId);
-            let remainingRooms = rooms.map(r => r);
+            const remainingRooms = rooms.map(r => r);
-            let allowedRoomsInThisTag = [];
+            const allowedRoomsInThisTag = [];
-            let lastFilterPriority = orderedFilters[0].relativePriority;
+            for (const filter of filters) {
-            for (const filter of orderedFilters) {
-                if (filter.relativePriority !== lastFilterPriority) {
-                    // Every time the filter changes priority, we want more specific filtering.
-                    // To accomplish that, reset the variables to make it look like the process
-                    // has started over, but using the filtered rooms as the seed.
-                    remainingRooms = allowedRoomsInThisTag;
-                    allowedRoomsInThisTag = [];
-                    lastFilterPriority = filter.relativePriority;
-                }
                 const filteredRooms = remainingRooms.filter(r => filter.isVisible(r));
                 for (const room of filteredRooms) {
                     const idx = remainingRooms.indexOf(room);
@@ -350,6 +351,7 @@ export class Algorithm extends EventEmitter {
         const allowedRooms = Object.values(newMap).reduce((rv, v) => { rv.push(...v); return rv; }, <Room[]>[]);
         this.allowedRoomsByFilters = new Set(allowedRooms);
         this.filteredRooms = newMap;
+        if (this.updatesInhibited) return;
         this.emit(LIST_UPDATED_EVENT);
     }
 
@@ -404,6 +406,7 @@ export class Algorithm extends EventEmitter {
             if (!!this._cachedStickyRooms) {
                 // Clear the cache if we won't be needing it
                 this._cachedStickyRooms = null;
+                if (this.updatesInhibited) return;
                 this.emit(LIST_UPDATED_EVENT);
             }
             return;
@@ -446,6 +449,7 @@ export class Algorithm extends EventEmitter {
         }
 
         // Finally, trigger an update
+        if (this.updatesInhibited) return;
         this.emit(LIST_UPDATED_EVENT);
     }
 
@@ -512,7 +516,12 @@ export class Algorithm extends EventEmitter {
         if (isNullOrUndefined(rooms)) throw new Error(`Array of rooms cannot be null`);
         if (!this.sortAlgorithms) throw new Error(`Cannot set known rooms without a tag sorting map`);
 
-        console.warn("Resetting known rooms, initiating regeneration");
+        if (!this.updatesInhibited) {
+            // We only log this if we're expecting to be publishing updates, which means that
+            // this could be an unexpected invocation. If we're inhibited, then this is probably
+            // an intentional invocation.
+            console.warn("Resetting known rooms, initiating regeneration");
+        }
 
         // Before we go any further we need to clear (but remember) the sticky room to
         // avoid accidentally duplicating it in the list.

src/stores/room-list/filters/CommunityFilterCondition.ts

@@ -1,5 +1,5 @@
 /*
-Copyright 2020 The Matrix.org Foundation C.I.C.
+Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -15,7 +15,7 @@ limitations under the License.
 */
 
 import { Room } from "matrix-js-sdk/src/models/room";
-import { FILTER_CHANGED, FilterPriority, IFilterCondition } from "./IFilterCondition";
+import { FILTER_CHANGED, FilterKind, IFilterCondition } from "./IFilterCondition";
 import { Group } from "matrix-js-sdk/src/models/group";
 import { EventEmitter } from "events";
 import GroupStore from "../../GroupStore";
@@ -39,9 +39,8 @@ export class CommunityFilterCondition extends EventEmitter implements IFilterCon
         this.onStoreUpdate(); // trigger a false update to seed the store
     }
 
-    public get relativePriority(): FilterPriority {
+    public get kind(): FilterKind {
-        // Lowest priority so we can coarsely find rooms.
+        return FilterKind.Prefilter;
-        return FilterPriority.Lowest;
     }
 
     public isVisible(room: Room): boolean {

src/stores/room-list/filters/IFilterCondition.ts

@@ -1,5 +1,5 @@
 /*
-Copyright 2020 The Matrix.org Foundation C.I.C.
+Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -19,10 +19,19 @@ import { EventEmitter } from "events";
 
 export const FILTER_CHANGED = "filter_changed";
 
-export enum FilterPriority {
+export enum FilterKind {
-    Lowest,
+    /**
-    // in the middle would be Low, Normal, and High if we had a need
+     * A prefilter is one which coarsely determines which rooms are
-    Highest,
+     * available for runtime filtering/rendering. Typically this will
+     * be things like Space selection.
+     */
+    Prefilter,
+
+    /**
+     * Runtime filters operate on the data set exposed by prefilters.
+     * Typically these are dynamic values like room name searching.
+     */
+    Runtime,
 }
 
 /**
@@ -39,10 +48,9 @@ export enum FilterPriority {
  */
 export interface IFilterCondition extends EventEmitter {
     /**
-     * The relative priority that this filter should be applied with.
+     * The kind of filter this presents.
-     * Lower priorities get applied first.
      */
-    relativePriority: FilterPriority;
+    kind: FilterKind;
 
     /**
      * Determines if a given room should be visible under this

src/stores/room-list/filters/NameFilterCondition.ts

@@ -1,5 +1,5 @@
 /*
-Copyright 2020 The Matrix.org Foundation C.I.C.
+Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -15,7 +15,7 @@ limitations under the License.
 */
 
 import { Room } from "matrix-js-sdk/src/models/room";
-import { FILTER_CHANGED, FilterPriority, IFilterCondition } from "./IFilterCondition";
+import { FILTER_CHANGED, FilterKind, IFilterCondition } from "./IFilterCondition";
 import { EventEmitter } from "events";
 import { removeHiddenChars } from "matrix-js-sdk/src/utils";
 import { throttle } from "lodash";
@@ -31,9 +31,8 @@ export class NameFilterCondition extends EventEmitter implements IFilterConditio
         super();
     }
 
-    public get relativePriority(): FilterPriority {
+    public get kind(): FilterKind {
-        // We want this one to be at the highest priority so it can search within other filters.
+        return FilterKind.Runtime;
-        return FilterPriority.Highest;
     }
 
     public get search(): string {

src/stores/room-list/filters/SpaceFilterCondition.ts

@@ -17,7 +17,7 @@ limitations under the License.
 import { EventEmitter } from "events";
 import { Room } from "matrix-js-sdk/src/models/room";
 
-import { FILTER_CHANGED, FilterPriority, IFilterCondition } from "./IFilterCondition";
+import { FILTER_CHANGED, FilterKind, IFilterCondition } from "./IFilterCondition";
 import { IDestroyable } from "../../../utils/IDestroyable";
 import SpaceStore, {HOME_SPACE} from "../../SpaceStore";
 import { setHasDiff } from "../../../utils/sets";
@@ -32,9 +32,8 @@ export class SpaceFilterCondition extends EventEmitter implements IFilterConditi
     private roomIds = new Set<Room>();
     private space: Room = null;
 
-    public get relativePriority(): FilterPriority {
+    public get kind(): FilterKind {
-        // Lowest priority so we can coarsely find rooms.
+        return FilterKind.Prefilter;
-        return FilterPriority.Lowest;
     }
 
     public isVisible(room: Room): boolean {
@@ -46,12 +45,7 @@ export class SpaceFilterCondition extends EventEmitter implements IFilterConditi
         this.roomIds = SpaceStore.instance.getSpaceFilteredRoomIds(this.space);
 
         if (setHasDiff(beforeRoomIds, this.roomIds)) {
-            // XXX: Room List Store has a bug where rooms which are synced after the filter is set
-            // are excluded from the filter, this is a workaround for it.
             this.emit(FILTER_CHANGED);
-            setTimeout(() => {
-                this.emit(FILTER_CHANGED);
-            }, 500);
         }
     };
 

test/autocomplete/QueryMatcher-test.js

@@ -183,18 +183,4 @@ describe('QueryMatcher', function() {
         expect(results.length).toBe(1);
         expect(results[0].name).toBe('bob');
     });
-
-    it('Matches only by prefix with shouldMatchPrefix on', function() {
-        const qm = new QueryMatcher([
-            {name: "Victoria"},
-            {name: "Tori"},
-        ], {
-            keys: ["name"],
-            shouldMatchPrefix: true,
-         });
-
-        const results = qm.match('tori');
-        expect(results.length).toBe(1);
-        expect(results[0].name).toBe('Tori');
-    });
 });

test/components/views/rooms/RoomList-test.js

@@ -296,6 +296,11 @@ describe('RoomList', () => {
             GroupStore._notifyListeners();
 
             await waitForRoomListStoreUpdate();
+
+            // XXX: Even though the store updated, it can take a bit before the update makes
+            // it to the components. This gives it plenty of time to figure out what to do.
+            await (new Promise(resolve => setTimeout(resolve, 500)));
+
             expectRoomInSubList(otherRoom, (s) => s.props.tagId === DefaultTagID.Untagged);
         });