WO2015092489A1

WO2015092489A1 - Method of controlling a graphical user interface for a mobile electronic device

Info

Publication number: WO2015092489A1
Application number: PCT/IB2013/061167
Authority: WO
Inventors: Magnus Johansson; Petter Alexandersson
Original assignee: Sony Corporation
Priority date: 2013-12-20
Filing date: 2013-12-20
Publication date: 2015-06-25
Also published as: WO2015092530A1; EP3084585A1; US20150177979A1

Abstract

The present invention concerns a method of controlling a layout of media items (11) on a user interface (20) for a mobile electronic device (1) comprising a touch sensitive display (4).

Description

METHOD OF CONTROLLING A GRAPHICAL USER INTERFACE FOR A MOBILE ELECTRONIC DEVICE

TECHNICAL FIELD

The present invention relates to a method of controlling a graphical user interface of a mobile electronic device, and a mobile electronic device using the method for controlling its graphical user interface.

BACKGROUND ART

The use of electronic devices such as computers, mobile phones, and audio/video equipment has had enormous development in the world in the last decades. A lot of effort has been made in making smaller electronic devices, in particular for portable use, such as mobile phones, laptop computers, Personal Digital Assistants (PDA), and so on. Much help has been obtained from miniaturization of electronic components and development of more efficient batteries. Mobile phones are getting smaller and smaller and the size is generally considered to be an important factor for the end customer, as well as the need for simplifying the handling of the applications in mobile phones by making the applications more intuitive handled.

Regarding mobile devices, the end users have a number of conflicting requirements. Basically, devices should be as small and light-weight as possible, while providing more and more advanced functions, and have a user-friendly interface. There is also only so much space in an electronic device, and in order to be competitive the elements of devices must be carefully packaged. The compact size of e.g. mobile phones also means that the user interface, typically a display, is limited. However, recently, so called smart phones have been provided with larger displays in new models compared to the former ones. In order to access different functions and stored items, different types of menu systems are employed in combination with touch sensitive displays with more and more use of icons representing, e.g. a set of data, a program, media files, e.g. music or video files, pictures/images, or an action, in mobile phones. A graphical user interface (GUI) is often configured with a plurality of icons on a display, each icon representing a certain subject/item being informative and/or interactive. Clicking on one of the icons, or activation in any other way of the icon, using the user input interface of the portable electronic device typically results in the presentation of a menu associated with the subject of the icon/item.

A problem related to the issue of generating GUI layouts for electronic devices, particularly portable devices such as mobile phones, is that the devices generally make use of one or more generic and fixed layouts for use in different themes. The layouts are typically hard coded into the software residing in the devices, and cannot be changed in an easy way. Furthermore, the layouts are often quite complex and a lot of work is generally required from both design engineers and software engineers before a final product is obtained. Today, one method used is to have design engineers create GUI layout documents, including sketches of the layouts with different associated measurements, such as offsets, heights and widths of objects and so on. This work is both cumbersome and time-consuming.

Another problem concerns that modern-day handheld communication devices, such as mobile phones, are capable of performing a multitude of tasks ranging from voice communication to sending e-mails, editing and sharing documents, browsing the Internet, play music and movie files, etc. However, due to the form factor of modern mobile communication devices many of these tasks are difficult to perform in parallel due to huge amounts of information presented on a restricted available area on touch sensitive displays of such devices. This compromise between a lot of information and a large number of icons on relatively small display areas is difficult to optimize as also the touch sensitive resolution of the touch sensitive display areas comprises a grid of a certain mesh that may not be too fine to create too much errors of detection when touching certain portions of the touch sensitive areas of the displays with a finger or stylus, e.g. by missing the desired area or touching a too large area. Even though the technological development leads to increased touch screen touch resolution and increased pixel density, there will still be a need for adapting the touch areas of interactive items on the screen to ensure that a user is able to identify and touch a desired item. Furthermore, the amount of functionalities expected to be performed by the user interface is steadily increasing which results in greater complexity for the user. The tedious navigation and operation of such devices may result in that users will be "lost" in different menus that do not appeal to them and/or menus that are too "rigidly" designed. This often also requires small fingers and a steady hand for any user of handheld electronic devices for not resulting in unnecessary and troublesome remake of faulty or undetected inputs and unsatisfactory long waiting time for the user.

SUMMARY OF THE INVENTION

With the above description in mind, then, an aspect of the present invention is to provide a method for control of a layout of media items and their media fields on an user interface for a mobile electronic device comprising a touch sensitive display, which seek to mitigate, alleviate, or eliminate one or more of the above-identified deficiencies in the art and disadvantages singly or in any combination.

These and further objects are achieved by a method for controlling a layout of media items on an user interface for a mobile electronic device comprising a touch sensitive display, the method comprising detecting a dual touch action comprising two touch points; determining a distance and direction of a movement of the two touch points in relation to each other;

changing the layout of the media items on the user interface in response to the determined distance and direction of the relative movement, and adapting of media fields corresponding to the media items on the touch sensitive display, wherein appearance and content of each media field corresponds to the layout of the media items.

In accordance with one aspect of the method above, each layout of media items comprises a predetermined grid of the media items on the user interface, whereby a current grid of the layout of the media items is changed to a different grid of the layout of the media items in response to the determined distance and direction of the relative movement of the two touch points. According to another aspect of the method above, each change of layout of media items is performed by means of a distinct snap-like switch from a current layout to a new layout in response to the determined distance and direction of the relative movement of the two touch points. In yet another aspect of the method above, each change of layout of media items is performed by means of switch from a current layout to a new layout by means of the dual touch action being either a pinch gesture on the touch sensitive display decreasing the distance between the two touch points or a spread gesture on the touch sensitive display increasing the distance between the two touch points, whereby a pinch changes to a finer grid of media items and a spread changes to a more coarse grid of media items. According to still another aspect of the method above, at least one layout of the media items comprises a coarse grid for a first media item and a fine grid for the remaining media items. In accordance with an additional aspect of the method above, the number of changes of layout of the media items is controlled in response to the determined distance and direction of the relative movement of the two touch points in one and the same dual touch action. According to yet an additional aspect of the method above, the method optimizes the appearance and content of each media field in relation to the size of each media item. According to a further aspect of the method above, the method optimizes the appearance and content of each media field in relation to a touch resolution of the touch sensitive display. In accordance with another aspect of the method above, the method optimizes the appearance and content of each media item in relation to a touch resolution of the touch sensitive display.

According to still another aspect of the method above, the amount of content of the media field increases when the distance between the two touch points increases in relation to each other and the amount of content of the media field decreases when the distance between the two touch points decreases in relation to each other. According to yet another aspect of the method above, the content of each media field comprises meta data and/or user selectable interactive action items, such as play or menu buttons or the like.

In accordance with additional aspects of the method above, at least one media field at least partly overlays at least one media item or at least one media field does not overlay any media item.

The invention also concerns a mobile electronic device using the method according to any preceding aspect above for controlling its user interface on a touch sensitive display.

The inventive method and mobile electronic device for control of a Pinch-Spread Layout Grid- User Interface (PSLGUI) allows users of media applications in portable electronic devices to quickly change and to set a layout of a display that suites them in the moment, e.g. when running, at the bus, in the car, or at work. One important factor is that the amount of information and actions shown via a layout of media items of a display on a portable electronic device can be made more flexible, user friendly and appealing to a larger group of users of the portable electronic device. Hence, each media item may show more or less accessible and optional amounts of information and/or interactively accessible links, icons and/or symbols than hitherto in media fields that are easy to change in size but also easy to reveal or hide depending on the used gesture of the user in more levels of information but also in more adaptable levels of information by using a simple and quick gesture. This is achieved due to the possibility of adapting each item, i.e. the media item and its media field, to the factual touch resolution of the display of the portable electronic device. This is done by making the media item and/or its media field larger for reaching over at least one touch sensitive square of the touch sensitive area of the display up to at least 4 to 5 touch sensitive square or up to all touch sensitive squares of the display if the media item and its media field covers the whole display. The media item and/or its media field may also reach over at least one and half touch sensitive square of the touch sensitive area of the display up to at least 4 to 5 touch sensitive square or up to all touch sensitive squares of the display if the media ite m and its media field covers the whole display. This enables for more accurate finger and/or stylus touching of the display but also enlarges the available area of interactive and/or non- interactive data disclosed for each media item/field, while enabling easy and quick hiding of the some or all the interactive and/or non-interactive data disclosed for each media item/field for a quick overview of all media items. This also enables a quick and easy "jump" from one current layout of media items to another layout of media items, e.g. by switching between one lay out or lay out level at a time or by switching/"jumping" past two or more layouts or lay out levels in one gesture or go. A further benefit is the quick and intuitive switching between layouts and associated touch areas of media items and media fields which is useful to a user since the demands on the touch area for the media fields and media items vary significantly depending on the situation. For instance if the user is exercising, it might be preferred to easily be able to change, by a single gesture, to a layout with larger touch areas for each media item and media field.

The layout change affects the sizes and structure of the media items on the touch sensitive display, as well as the amount of information and actions shown on each item by means of more accessible and choosable levels of information than hitherto. The advantage is that one gesture being either a pinch or a spread may change from one level/layout of information and/or interactive media items with an overview of media items with no media fields visible or accessible to another level with the smallest media field visible with for example only one row of information in one step. In another example, this one gesture being either a pinch or a spread may change from one level/layout of information and/or interactive media items with an overview of many media items (e.g. in a 4 x 4 grid of media items of the same size, see Fig 4 and corresponding Fig. 4A with this grid shown with dash-dotted lines as a grid G) with no media fields visible or accessible to another level with a small media field. Such a different lay out level is shown in Figs. 3, 5, 5A, 6, 6A, 7, 8, 9, and 10 (e.g. in a 3 x 3 grid of media items of the same size or with one media item being larger, e.g. media item 11 is larger in Figs 5, 5A, 8, and 10 than the other media items 12-16 and Fig. 5A corresponding to Fig 5 with a grid shown with dash-dotted lines as a grid G). Here, the media field is larger than the smallest media field times two or three or more with for example two or three or more rows of information and/or interactive links, icons and/or symbols visible on the associated media item. Another level of laid out media items 10, 11-16, X being possible to change or switch into according to the invention is shown in Figs. 6 and 7. Fig 6 discloses a layout of media items 11-14 of the same size in a 2 x 2 grid, but, here, one or more media items could be smaller or larger than the other ones, see also corresponding Fig. 6A with the touch grid shown with dash-dotted lines as a grid G. Fig 7 discloses a layout of media items 11-12 of the same size in a 1 item row grid with a width over the whole breadth of a display, but, here, one or the other or even more media items could also be smaller or larger than the other ones, e.g. with a smaller or larger height and/or a smaller or larger width.

This touch grid of the is referred to as the grid G in Figs. 4A, 5A, and 6A, whereby media item 11 is the smallest (one touch square) in the largest layout of media items 11-16, X of Fig 4A and when switching by spreading to the next layout of Fig 5A, the first media item 11 gets bigger than the remaining ones 12-16 and media item X is hidden or "disappears", but each media item 11-16 is now shown with a media field 30 of which media item 11 has the largest media field with somewhat more data/information contained than the other media fields. Hence, the now shown media fields 30 of Fig. 5A extends of more than one touch square of the grid G instead of the media items 11-16, X in Fig. 4A covering only one touch square of grid G. This is even more evident when comparing Figs. 6 and 6A, where the media fields 30 for each media item 11 and 12 covers two touch squares of the touch grid G when switched from the layout of media items in Figs. 5 and 5A with smaller media fields 30 and correspondingly less media field content. This is also applied to Fig. 7 where even larger media fields 30 of media items 11 and 12 are revealed after switching from the layout of Figs. 6 and 6A to the layout of Fig 7 by spreading gesture revealing even more content of each media field 30. Hence, when pinching, the layout may be changed back from the one in Fig 7 to the one in Fig 6/6A and then to the one in Figs. 3, 5/5A, 8, 10 and then at last to the layout of Figs. 4/4A, and 11. However, a larger distance D, Dl, D2 when pinching or spreading may switch from the layout in Fig 7 to the layouts of Figs. 5/5A or 4/4A in one snap-like switch without passing the intermediate ones.

One example is based on the Pinch-Spread Layout Grid-User Interface (PSLGUI) of a current music album application where a grid laid-out user interface (Ul) is easily changed by users in regard of media item grid size and item position by performing a pinch or spread action.

The inventive idea combines this PSLGUI with information and action-layering meaning that when users zooms in, and thereby enlarges the media items, the more information as well as more actions on the media items become accessible as explained above. When the user zooms out less information and/or no actions are shown on the media items. Possibly, a fully zoomed out mode could show only a summary of the items in the grid.

The uniqueness of this innovation according to the invention lies in the combination of a learnable gesture (pinch-spread by at least two fingers) with a principle of information layering, in the context of media presentation enabling more or less information and/or interactive actions to be accessible for each media item.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the present invention will be more apparent from the following description of preferred embodiments with reference to the accompanying drawings, in which

Fig. 1 schematically illustrates an electronic device in the form of a mobile phone, configured to be operated in accordance with the invention;

Fig. 2 schematically illustrates an electronic device in the form of a mobile phone, configured to be operated in accordance with the invention; Fig. 3 schematically illustrates one lay out of a graphical user interface (GUI) on a touch sensitive display of the mobile phone of Figs. 1 and 2 with media items whose sizes are defined by white areas similar to white lines;

Figs. 4 to 6 show other GUI layouts that may be changed between for use as GUI:s on the touch sensitive display in the mobile phone of Figs. 1 to 3;

Figs. 4A to 6A correspond to Figs. 4 to 6 and show other GUI layouts that may be changed between for use as GUI:s on the touch sensitive display in the mobile phone of Figs. 1 to 3 but also show a grid in dotted lines associated with the touch resolution of the touch sensitive display for disclosing the relationship between sizes of media items and their media fields and the touch grid according to the invention;

Figs. 7 to 9 illustrate yet other GUI layouts that may be changed between for use as GUI :s on the touch sensitive display in the mobile phone of Figs. 1 to 6A;

Fig. 10 and 11 also illustrate further GUI layouts that may be changed between for use as GUI:s on the touch sensitive display in the mobile phone of Figs. 1 to 9 for the interactive change between different entities in the process of generating a GUI layout on the display of the mobile phone, by means of the invention.

Fig. 12 illustrates a flowchart of a method for controlling the GUI :s of the touch display in the mobile phone according to the invention.

DETAILED DESCRIPTION

Embodiments of the present invention relate, in general, to a radio communication terminal, such as a mobile phone, including one or more marker input devices. Embodiments of the present invention relate, in general, to the field of automatically launching different layouts of applications in mobile communication devices. However, for the sake of clarity and simplicity, most embodiments outlined in this specification are related to mobile phones. However, it should be appreciated that the invention is as such equally applicable to electronic devices which do not include any radio communication capabilities.

In the following description reference is made to the accompanying drawings. In this regard entity, dimensional, positional and directional terminology, such as "top", "bottom", "front", "back", "first", "second", "overlaid", "non-overlay", "first/second/top/bottom layer", "larger", "smaller", "finer", "more coarse", etc, is used with reference to the size, resolution, position and orientation of the objects and figures being described. Because components and objects/items of embodiments of the present invention can be positioned in a number of different orientations and have a varying order of use and placing, be of different sizes and resolutions, the terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present protection. The following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

Exemplary embodiments will now be described with references made to the accompanying drawings. Figures 1 to 11 illustrates a mobile electronic device 1, i.e. at least its display. Fig 12 schematically illustrates a method according to the invention for controlling and adapting a graphical user interface of the mobile electronic device 1 to the likings of a user in regard of changeable lay outs of media items 10 on a touch sensitive display 4 of the mobile electronic device 1. Examples of devices 1 are mobile phones, smart phones, laptops, touch pads etc.

Figs. 1 and 2 schematically illustrate the portable electronic device 1 in the form of a mobile phone. Details of the mobile phone 1 as such are of little importance to the invention, but for the sake of clarity it should be mentioned that the phone 1 comprises a support structure 2 including a chassis and a cover, directly or indirectly supporting the other components of the phone. Phone 1 is further devised with a user interface comprising a keypad 3 and/or the touch-sensitive display 4, as well as a microphone 5 and at least one speaker 6. Though not shown in Figs. 1 and 2, phone 1 further comprises a signal transceiver, including an antenna and radio transmission and reception electronics, for communicative connection to base stations of communications networks, such as GSM or WCDMA networks; a power supply in the form of a battery, and a cable socket for connection to a mains outlet via a transformer. Phone 1 is also devised with a computer system, including a microprocessor with associated memory and software, configured to carry out the tasks of signal processing and data handling in the phone. All of these features are, as such, well known in the field of telecommunication devices but not all shown on the figures. The phone 1 comprises a processing unit 7 that may be constituted by any suitable Central Processing Unit, CPU, microcontroller, Digital Signal Processor, DSP, etc., capable of executing computer program code for operating phone 1. A software handler may be implemented utilizing the processing unit 7 for determination and switching between different input and/or user interface modes and/or lay outs of media items 10. The phone 1 further comprises a memory 8 that can be used for storing computer program code, data code in files representing e.g. images, games, music, and video, and other relevant or desired data etc. The memory 8 can be any combination of a Read and write Memory, RAM, and a Read Only Memory, ROM. The memory 8 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, or solid state memory or even remotely mounted memory. The phone 1 also comprises other components required for its operation, such as input buttons at its support structure 2, circuit wiring and associated components, but these components are not explained further as this is common knowledge for a skilled person.

The control unit 7 represents the microprocessor of the computer system configured to control the processes carried out within the mobile phone 1. In order for a user to control and monitor the processes carried out under control of the control unit 7, all the above elements making up the phone 1 are operatively and communicatively connected to the control unit 7. In the present embodiment, the control unit 7 is also connected to the radio communication components for wireless communication, but these enabling components have no reference numerals as these components are known and any further details of them have no importance in regard of understanding the invention.

In accordance with the invention, at least one of the problems above is solved or at least reduced by combining the detection of a dual touch action comprising two touch points I, II with simultaneous determination of a command associated with a distance D, Dl, D2 and direction of the dual touch action (see black filled arrows shown with unbroken lines of Figs. 3 - 11) of in regard of a movement of the two touch points I, II in relation to each other when entered on the touch-sensitive display 4, as a trigger to execute an desired input action in the mobile phone 1. This desired input action comprises changing the layout of media items 10 on the user interface 20 of the mobile phone 1 in response to the determined distance and direction of the relative movement. This change of layout of media items 10, 11-16 also comprises adapting of media fields 30 corresponding to the media items on the touch sensitive display 4, such that the appearance and content of each media field 30 corresponds to the layout of the media items. This change from one layout of media items 10, 11-16, X to another is shown on Figs 3 - 11 (symbolized by black arrows shown with dotted lines and nonfilled arrow heads) in these figures for a better understand of switching between the layouts.

Figs. 2 - 11 illustrate a scenario according to an embodiment of the invention, where phone 1 is placed in action mode and/or in operational mode ready for use. The phone 1 may contain a Near Field Communication interface, e.g. Bluetooth™, but this will not be described in any greater detail. The embodiment of Figs. 2 to 11 relates to a scenario where the user of the phone 1 wants to handle at least one media item 10, 11-16, e.g. a media file, such as music, video or a picture or the like. A plurality of objects 11-16 (one item shown with X in Fig. 2 to visualize that more than six, i.e. 11-16, seven and eight media items 10 may be present on the display 4 depending on application running on the phone 1 and the wishes of its user), in this case media items in the form of pictures of e.g. music albums, are shown on the display 4. In the illustrated embodiment there are at least one or more media items 10-16 being music albums shown. The displaying of these media items 10-16 may be activated responsive to the dual touch action, wherein displaying of the media items 10-16 stored in a memory 9 in a certain layout is set as a default action to be carried out when the type and size of the dual touch action is detected. Alternatively, a command has been entered by the user of phone 1, by means of an input key of keypad 3 or on touch-sensitive display 4, to display all media items 10-16 of memory 9, or all media items of a certain folder in memory 9, i.e. only certain media items, e.g. only media items 11-16 in Fig. 3 or only media items 11-14 in Fig. 6 or only media items 11 and 12 in Fig. 7. The media items 10 may be shown in the same sizes but also be shown in different sizes or be shown with at least one media item, such as media item 11 in Figs. 3, 5, 5A, 8, 9, in a larger state than the remaining items 12 - 16.

In a state of the art mobile phone, there are a number of different actions that may be carried out when one or more media items are shown on display 4. Using a soft or touch key function, you may e.g. edit a picture, rename it, use it as background, send it and so on. In the present embodiment, though, the simultaneous presentation of the media items 10 and detection of an established or predetermined dual touch action, renders a predetermined executable action such as a layout change to be ready to be carried out or in fact carried out, and preferably also presentation of a new layout on the touch display 4 indicating/showing the result of the action, namely a greater or lesser resolution of media items 10 on the display. By touching the first media item 11 or object (or a second object 12, a third object 13 or a fourth object 14, or a fifth object 15, or a sixth object 16 , and so on), e.g. a music or video file, or a picture, on display 4, the dual touch action interface is triggered to execute the desired command based on that object after detecting the direction of the movement of the two touch points I, II in relation to each other, either a pinch or spread direction, i.e. to change the layout of the media items into a finer grid of media items (the pinching effect) or to a more coarse grid of media items (the spreading effect), e.g. to a greater or lesser number of music album icons or pictures on the display. The distance of the movement of the two touch points I, II in relation to each other determines the scale or size of the resolution, i.e. the degree of the fineness and coarseness of the layout of media items 10, 11-16, X changed or switched into or to be changed or switched into after the performance and execution of the command associated with the specific dual touch action gesture.

Specifically, the method according to the invention for controlling a layout of media items 10, 11-16, X on a user interface 20 for the mobile electronic device 1 comprising the touch sensitive display 4 having a specific touch resolution or touch grid G. This touch resolution/grid G may be about 9 x 9 mm per square times the number of squares for the whole touch sensitive area/grid of the display 4. This dimension of 9 x 9 mm for each touch square may be larger or smaller depending on the display technology used or the actual resolution demands for users or manufacturers or of the specific type or size of the portable electronic device. The method specifically comprises detecting a dual touch action comprising two touch points I, II . The method also comprises determining a distance D, Dl, D2 (see Figs 3 - 5, and 7 - 9) and direction of a movement of the two touch points I, II in relation to each other (see direction of relative movement of touch points I and II symbolized by arrows shown with unbroken lines and filled arrow heads of Figs 4, 4A, 5, 5A, and 7 - 9). The touch areas or points I and II are also symbolized as circular areas shown with dotted lines in Fig. 3. The method also comprises changing the layout of the media items 10, 11-16, X on the user interface 20 in response to the determined distance D, Dl, D2 and direction of the relative movement between the two touch points I, II in relation to each other. The method also comprises adapting of media fields 30 corresponding to the media items on the touch sensitive display 4, so that the appearance and content of each media field corresponds to the desired or chosen layout of the media items. The method also comprises that each layout of the media items 10, 11-16, X in turn comprises a predetermined grid of the media items on the user interface 20, i.e. enabling that a current grid of the layout of the media items is changed to a different grid of the layout of the media items in response to the determined distance D, Dl, D2 and direction of the relative movement of the two touch points I, II. This may be done by performing a distinct snap-like switch from a current layout to a new layout in response to the determined distance D, Dl, D2 and direction of the relative movement of the two touch points I, II. This snap-like switch creates an intuitive handling and effect of the pinch or spread gesture as it is immediately and directly recognizable in a very simple way.

Moreover, each change of layout of the media items 10, 11-16, X is performed by means of switch from the current layout to any new layout by means of the dual touch action being either the pinch gesture on the touch sensitive display 4 decreasing the distance D, Dl, D2 between the two touch points I, II (see Figs. 10 and 11 showing the increased distance of D, Dl, D2 as general increase by symbol D >) or a spread gesture on the touch sensitive display 4 increasing the distance D, Dl, D2 between the two touch points I, II (see Figs. 10 and 11 showing the decreased distance of D, Dl, D2 as general increase by symbol D <).

Hence, the pinch gesture changes to a finer grid of the media items 10, 11-16, X, enlarging the size of the associated media fields 30 for revealing more and more amount of content until each media field cannot be any larger, revealing a maximum content of at least one media field 30 if not all media fields shown are of the same size at the "lowest" or most enlarged or zoomed in level of media item layouts.

The spread gesture changes to a more coarse grid of the media items 10, 11-16, X, lessening the size of the associated media fields 30 for revealing less and less amount of content until at least one media field of at least one media items 10, 11-16, X disappears or is totally hidden, if not all media items are of the same size. Then, in the top or most zoomed out level of media item layouts and if all media items 10, 11-16, X are of the same size, all the media fields 30 revealed in the former layout have disappeared, i.e. are totally hidden. Hence, at least one layout of the media items 10, 11-16, X may comprise a more coarse grid for a first media item 11 and a finer grid for the remaining media items 12-16 (see Figs. 3, 5, 5A, 8, 9, 10). This means that the number of changes of layout of the media items 10. 11-16, X is controlled in response to the determined distance D, Dl, D2 and direction of the relative movement of the two touch points I, II in one and the same dual touch action.

Therefore, the method may optimize the appearance and content of each media field 30 in relation to the size of each media item 10, 11-16, X. The method may also optimize the appearance and content of each media field in relation to the touch resolution G of the touch sensitive display 4 and may optimize the appearance and content of each media item in relation to the touch resolution G of the touch sensitive display. Furthermore, if desired, the media field 30 may at least partly overlay any media item 10, 11-16, X. However, if at least one media item resembles any artwork, the media field 30 may not overlay this media item, or, if desired, the media field 30 may not overlay any media item.

The effect of the invention as explained above is that it provides all types of users, situations and devices with an easily adaptable dynamic layout of media items 10, 11-16, X on a touch sensitive display 4, whereby the same and static layouts in an application as in prior art applications is eliminated. This is due to that different users prefer different layout styles on their portable electronic devices 1. Some users prefer a display view with many small interactive items while others want few large items that are easy to tap. Other users require a lot of information when presented with a previously unknown item 10, 11-16, X in order to take action on it by tapping it, while others consider more text as messy and "in the way". Some users never use secondary action while for some they are the key to the whole experience of a more suitable layout satisfying functional and practical needs, but yet optical/graphical preference. Some users consume a lot of new content every day, and others are more selective. Hence, this inventive way of "picking"/changing into preferable layouts on displays of portable devices 1 fulfills the needs of a broader span of "mass consumers", but also more "picky" ones than a standard prior art grid layout does. There is also a need to adapt the layout to a current situation, e.g. when being on the move, for example running or walking, a layout with large touch targets/items 10 would be preferred. On the other hand - when the user has a lot of time on their hands they might be interested in reading more about each item 10, 11-16, X.

By combining a touch-sensitive display function with detection of a dual touch action on a portable communication device, the ease of use of the portable communication device and its different menus of its graphical user interface is enhanced. Making e.g. a plurality of selectable commands by a pinch gesture or a spread gesture on the touch sensitive display decreases the invisibility of interaction with the graphical user interface and the underlying software, e.g. applications and any media files/data, so that the user will sure of what actually happens in the interaction with the graphical user interface as it is more clearly visible. Furthermore, there will be fewer accidental interactions when the mobile phone is positioned in for example a pocket or bag, since the user needs to use a dual touch action on an object on the display/screen to trigger a desired action. This increases the control over the near field communication process. Furthermore, the present invention provides the ease of use sought for by the market, in a time where different types of data, software, and advanced graphical interfaces, e.g. for playing music or video or interacting with different social media, and of course for the wireless communication with other mobile phones or the like wireless devices, not only provide extended capabilities for a user but also requires a higher degree of knowledge from the users.

Figure 12 illustrates a flowchart of the method for controlling the input mode of the touch display/screen 4 for the phone 1. The phone comprises at least one input detection system, e.g. a capacitive touch detection system (not shown) indicating touching on the touch display 4. This capacitive touch detection system indicating touching on the touch display 4 is operatively connected to the processing unit 7 of the phone 1 for enabling the control of the GUI 20 according to the invention.

The pinch and spread gestures according to the invention achieves an information layering in combination with a "surfacing" or hiding of secondary actions or more detailed information in the distinct snap-like switch from a current layout state to a new layout state of media items on mobile devices 1. Each layout state can be seen as a separate zoom level with its own unique purpose. How many laid out states or zoom levels that are used or passed from one layout to another depends on and are represented by the size of the gesture, i.e. the distance D, Dl, D2 between the two touch points I, II in relation to each other. The distance may be measured in relation to the centres of the touch points or areas I or II (see D and Dl of Fig. 3) or in relation to the outer circumference of the touch points or areas I or II (see D2 of Fig. 3). As an example, a pinch gesture may be used to hide or re-position titles on music albums and a spread gesture increasing the distance D, Dl, D2 between points I, II can show titles and if spread more, i.e. if D, Dl or D2 gets larger (D >), more information, e.g. more rows of information or larger areas and number of interactive icons, such as play buttons, stars/cookies/favourites, is shown. The pinch gesture as used for hiding or re-positioning media items 10 and their media fields 30 decreases the distance D, Dl, D2 between points I, II, and if pinched more, i.e. if D, Dl or D2 gets smaller (D <), less information, e.g. less rows of information or smaller areas and number of interactive icons, such as play buttons, stars/cookies/favourites, is shown until when fully pinched out, no information is shown except for pictures.

Each change of layout of media items 10, 11-16, X is performed by means of the distinct spring-/leap-/snap-like changeover/transition from a current layout to a new layout in response to the determined distance D, Dl, D2 and direction of the relative movement of the two touch points I, II.

The number of interactive media fields is easy to adapt to the corresponding media items on the touch sensitive display, wherein the number of media items corresponds to the determined distance and direction of the relative movement. At least one media item may also be detected as a centre point or area corresponding to the two touch points, such that the new layout that is changed into centers the at least one media item. As explained earlier, the layout of at least one media item on the GUI may be changed in response to the determined distance over at least two levels of layers or layouts instead of only one layout or layout level at a time depending on the actual pinch/spread size.

Information and media items and media fields (contents thereof) may not only be limited to the sharing of music, video or picture files. Other type of information may also be shared such as streaming data, links or references to data, contact information, message data, text and document data (e.g. .txt, .pdf, .doc, .ppt, .exe, etc.), and metadata. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" "comprising," "includes" and/or "including" when used herein in this description and in the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. The foregoing has described the principles, preferred embodiments and modes of operation of the present invention. However, the invention should be regarded as illustrative rather than restrictive, and not as being limited to the particular embodiments discussed above. The different features of the various embodiments of the invention can be combined in other combinations than those explicitly described. It should therefore be appreciated that variations may be made in those embodiments by those skilled in the art without departing from the scope of the present invention as defined by the following claims.

Claims

A method of controlling a layout of media items (10, 11-16, X) on an user interface (20) for a mobile electronic device (1) comprising a touch sensitive display (4), the method comprising:

- detecting a dual touch action comprising two touch points (I, II),

- determining a distance (D, Dl, D2) and direction of a movement of the two touch points (I, II) in relation to each other,

- changing the layout of the media items (10, 11-16, X) on the user interface (20) in response to the determined distance (D, Dl, D2) and direction of the relative movement, and

- adapting of media fields (30) corresponding to the media items on the touch sensitive display (4), wherein appearance and content of each media field (30) corresponds to the layout of the media items (10, 11-16, X).

A method according to claim 1, wherein each layout of the media items (10, 11-16, X) comprises a predetermined grid of the media items on the user interface (20), whereby a current grid of the layout of the media items is changed to a different grid of the layout of the media items in response to the determined distance (D, Dl, D2) and direction of the relative movement of the two touch points (I, II).

A method according to claim 1 or 2, wherein each change of layout of the media items (10, 11-16, X) is performed by means of a distinct snap-like switch from a current layout to a new layout in response to the determined distance (D, Dl, D2) and direction of the relative movement of the two touch points (I, II).

A method according to claim 3, wherein each change of layout of the media items (10, 11-16, X) is performed by means of switch from a current layout to a new layout by means of the dual touch action being either a pinch gesture on the touch sensitive display (4) decreasing the distance (D, Dl, D2) between the two touch points (I, II) or a spread gesture on the touch sensitive display (4) increasing the distance (D, Dl, D2) between the two touch points, whereby a pinch changes to a finer grid of media items and a spread changes to a more coarse grid of media items.

5. A method according to claim 4, wherein at least one layout of the media items (10, 11- 16, X) comprises a coarse grid for a first media item (11) and a fine grid for the remaining media items (12-16, X).

6. A method according to any preceding claim, wherein the number of changes of layout of the media items (10, 11-16, X) is controlled in response to the determined distance (D, Dl, D2) and direction of the relative movement of the two touch points (I, II) in one and the same dual touch action.

7. A method according to any preceding claim comprising optimizing the appearance and content of each media field (30) in relation to the size of each media item (10, 11-16, X).

8. A method according to any preceding claim comprising optimizing the appearance and content of each media field (30) in relation to a touch resolution (G) of the touch sensitive display (4).

9. A method according to any preceding claim comprising optimizing the appearance and content of each media item (10, 11-16, X) in relation to a touch resolution (G) of the touch sensitive display (4).

10. A method according to any preceding claim, wherein the amount of content of the media field (30) increases when the distance (D, Dl, D2) between the two touch points (I, II) increases in relation to each other and the amount of content of the media field decreases when the distance between the two touch points decreases in relation to each other.

11. A method according to claim 10, wherein the content of at least one media field (30) comprises meta data and/or user selectable interactive action items, such as play or menu buttons or the like.

12. A method according to any preceding claim, wherein at least one media field (30) at least partly overlays at least one media item (10, 11-16, X).

13. A method according to any of the claims 1 to 11, wherein at least one media field (30) does not overlay any media item (10, 11-16, X). A mobile electronic device (1) using the method according to any preceding claim controlling its user interface (20) on a touch sensitive display (4).