The Information Networks Laboratory is engaged in research that integrates information systems and networks to realize new network services.
Network services such as cell phone services, web conferencing, network games, agriculture, transportation, and various Internet of Things (IoT) applications are provided via networks and information is processed via networks.
In a broad sense, networks that connect computers and networks that connect users and computers are part of information processing systems.
Usually, network services are required to provide comfort with high-capacity and low latency, convenience for use from any location, and safety for use without concern for security risks.
In this age of high-performance computers and high-speed networks, we consider networks part of information processing systems and are engaged in research on information processing systems to realize such comfort, convenience, and security.

Network distributed processing architecture

One of the challenges of services provided over the network is latency. Even in network games, unfairness due to delay differences occurs depending on the distance from the information processing system in which the application is running. For example, if the application is running on a server in Tokyo, the user in Tokyo and the user in Okinawa will be disadvantaged in terms of delay. An information processing system that eliminates these unfairness requires a mechanism that minimizes the delay of all users, such as the placement of the information processing system in the network and the processing method of the application, and makes the delay fair. .. The research theme of the network distributed processing system deals with research issues that realize low delay and fair delay control of services provided via a wide area network, assuming various usage scenarios.

Network service processing system

The current network is operated and managed independently of the mobile network that provides mobile phone services, the fixed network that provides the Internet line at home, and the cloud where applications run. The user terminal also puts the application on the smartphone, puts the application on the personal computer at home, and the data is also saved in each terminal. For example, there are many challenges in realizing an environment in which the exact same environment can be instantly reproduced even if a game or web conference that was running on a smartphone can be continued on a home computer without stress, or even if a computer or smartphone is replaced. The research theme of network service processing system deals with research issues of network service processing system authentication method, data deployment method, and network architecture, which are safe to use with peace of mind and convenient to use regardless of location or device.

Ultra-low latency optical network

For a large-capacity, low-latency network environment, it is necessary to change the way the network is created. The current network is roughly a switching system that distributes information to each destination and a transmission system for transmitting information far away, and realizes an information transmission function. While switching systems allow fine-grained destination selection, they can also be a factor in increasing latency because they buffer information. One approach to achieving a high capacity, low latency network is to configure a full mesh network without the use of switching systems that can increase latency. However, with a simple full-mesh configuration, there are problems such as the number of connectable bases is limited and it is not possible to select destinations with fine particle size. The research themes of ultra-low latency optical networks deal with research themes such as layered networks that solve these problems and optical wavelength network construction methods that efficiently utilize a small number of wavelengths.

We are engaged in research and development on information security and cryptographic technologies that are required for secure communication and information processing on the Internet. Modern cryptographic technology is designed to be provably secure based on computational hard problems such as prime factorization problem and discrete logarithm problem, so theoretical analysis to prove its security is required. On the other hand, cryptographic technology is actually used in the Internet, so efficiency and practicality are also important. Based on these theoretical and practical viewpoints, we do research and development on information security, especially public key cryptography and cryptographic protocols, with the following research topics.

Cryptographic Protocol

We do research and development on cryptographic protocols, which realize advanced functions using cryptographic primitives, e.g., key exchange protocol TLS widely used in the Internet, and distributed ledger technology block chain used in Bitcoin etc. We design and implement secure and efficient cryptographic protocols.

Functional Encryption

We do research and development on functional encryption and other highly functional cryptographic primitives, e.g., ID-based encryption, and anonymous signature etc. We also study and implement efficient algorithms for elliptic curve and pairing, which is the basis of the functional encryption and highly functional cryptographic primitives.

2-party / Multi-party Computation

We do research and development on 2-party / multi-party computation, which is the key technology to utilize sensitive information, e.g., personal information and medical information, effectively, while keeping privacy. 2-party / multi-party computation is a “universal” cryptographic protocol that can compute arbitrary function on inputs of the parties, without revealing the inputs.

We have been studying on the electronic properties of biological macromolecules such as DNA, proteins and ligand using ab initio molecular simulations, in order to propose novel potent medicines for treating globally feared diseases such as Alzheimer’s disease, tuberculosis and cancer. In addition, to elucidate the transcription mechanism of genetic information from DNA to RNA, which is controlled efficiently by some transcriptional regulation proteins, we investigate the specific interactions between DNA and these proteins by the ab initio molecular simulations. Some of the key research themes of our laboratory are shown below.

Elucidation of aggregation mechanism of amyloid-beta (Aβ) proteins

Onset of Alzheimer’s disease is shown to be deeply associated with the formation of amyloid plaque in a patient’s brain. However, the formation mechanism of the plaque from Aβ proteins is not clarified yet. We are in an attempt to elucidate the mechanism, in order to propose novel compounds for preventing the formation of the plaque. These compounds might be potent inhibitors against Alzheimer’s disease.

Proposal for potent medicines against Alzheimer’s disease

We investigated the specific interactions between Aβ protein and many kinds of compounds and proposed novel inhibitors against the aggregation of Aβ s. These inhibitors are the derivatives of curcumin, which is included in natural product turmeric. Our collaborators in Ukraine are attempting to synthesize these derivatives, and they are expected to be potent medicines for Alzheimer’s diseases.

Proposal for potent medicines against tuberculosis

Tuberculosis (TB) is one of the most widespread infection diseases in the world, and many types of anti-TB drugs were developed. However, since Mycobacterium tuberculosis (Mtb) can easily get drug resistance, it is necessary to develop novel effective anti-TB drugs targeting the most conservative proteins. We employed the filamental temperature-sensitive Z (FtsZ) as the target protein and investigated the binding properties between FtsZ and many types of compounds using ab initio molecular simulations. Based on the results simulated, we proposed some novel compound inhibiting the polymerization of FtsZ proteins, which causes the cell division of Mtb.

Proposal for potent inhibitors against cancer metastasis

To develop more potent inhibitors against the cancer metastasis, we investigated the binding properties of amino acid peptides with a protein, which is considered to be a trigger for the onset of cancer metastasis. Based on the results simulated, some novel peptides having large binding affinity to the protein were proposed as potent inhibitors. Our proposed new peptides are expected to be new inhibitors against cancer metastasis.

Elucidation of the transcription mechanism controlled

by regulatory proteins The transcription mechanism of gene information from DNA to RNA is efficiently controlled by various regulatory proteins as well as ligand molecules. The catabolite activator protein (CAP) is one of the regulatory proteins, and the complex of CAP with cyclic AMP (cAMP) plays an efficient role in the transcription mechanism. We investigated the effect of cAMP-binding to CAP on the interactions between DNA and CAP using ab initio molecular simulations, in order to elucidate the transcription mechanism controlled by CAP and cAMP. The results will be helpful for proposing novel compounds controlling the transcription mechanism.

The Computer Systems Performance Engineering Laboratory studies performance of any platforms of computers from ultra high-speed supercomputers to extreme low-power mobile devices based on scientific and engineering approaches. Currently, many deep learning programs, which become a typical component for AI-based applications, has struggled with lack of performance even if it is implemented on the state-ofthe-art CPUs or GPUs. Therefore, techniques that realize high-performance and high-efficiency computer systems are expected to be an enabler of emerging new AI applications such as self-driving cars and autonomous intelligent robots. In the context of system performance matter, we highlight specialization for inherent memory access locality which will be a clue to its solution.

Memory-centric customization toward FPGA accelerators

For realizing a human-centered super smart society called Society 5.0, it is very important to attain highperformance and high-efficiency computing through custom computing systems specialized for particular application domains. On the other hand, customization of systems incurs challenges in productivity and costs for the hardware and software development process. To overcome these difficulty, we investigate the ways to detect locality of processing inherent in application programs and develop high-level optimization frameworks that automatically map it to FPGA accelerators.

Scientific modeling of system performance and quality of the performance

We investigate the ways to profile, analyze and predict behaviors of program execution to understand the software execution performance. Here, we develop profilers, performance models, and simulators of hierarchical memories.

Automation of customization and co-design driven by mathematical optimization and

machine learning techniques We investigate the ways to automate co-design processes for domain-specific customization. These are implemented upon the current software stack such as compilers, code translators, and optimizers. These are also strongly tackled with mathematical optimization, machine learning, and deep learning techniques.

My laboratory develops new methodologies and software applications on computational chemistry and chemoinformatics using the latest computational science and information technologies, such as highperformance parallel distributed programing, data assimilation using computer simulation and database, and machine learning with deep neural net technology. The main research themes are introduced as follows.

Exploring Molecular Conformation and Crystal Structure Polymorphism

Many of drug molecules and organic molecular materials are used as solid crystals, and their medicinal effects and material properties are determined by their stable crystal structure. However, in these molecular crystals, it is possible that a molecule in crystal may form different 3D structures (conformations) or different crystal structures (polymorphisms). Therefore, prediction of some stable crystal structures in advance is important information for designing pharmaceutical molecules and organic materials. Our computational chemistry application CONFLEX that can be exploring molecular conformations and crystal structure polymorphisms, is developed as a supporting tool for advanced molecular design.

Protein-Ligand Docking Simulation by using Coarse-Grained Potentials

A drug molecule binds to a target protein and forms a complex structure. As a result, information transmission in vivo is activated or inhibited, and the function expression of the protein is controlled. In order to elucidate essential biomolecular behaviors and biological phenomena in the fields of drug discovery, we are developing a new docking simulation method so that can efficiently search some plausible ligand binding sites by a coarse-grinded model and rapidly produce highly accurate all-atom model by data assimilation with conformation database.

Molecular Activity and Material Property Prediction by using Deep Neural Nets

Machine learning technology using deep neural nets has made amazing results in the field of speech recognition and image analysis. Furthermore, it is expected that paradigm shift will occur in structural activity correlation of drug molecules and prediction of physical-chemical properties for electronic device materials. Our laboratory formulates new molecular descriptors in terms of structures, energies, and electronic states based on computational chemistry simulation, and develops a next generation system on molecular activity and property prediction by using some deep learning platforms.

A distributed system consists of multiple nodes connected by a network. The nodes communicate with each other and cooperate to accomplish a given task. Because there are many nodes in a distributed system, failure of some nodes is inevitable. Also, the latest information might not be obtained due to network communication delays. While most information services we usually use are built as distributed systems, there are such difficulties in realizing the systems. Distributed Systems Laboratory aims to solve these problems and realize efficient distributed systems for various applications.

Byzantine Fault Tolerance

A failed node in a distributed system may behave unexpectedly. Also, if a malicious attacker takes over a node, it may interfere with the processing of other nodes. Such kinds of behavior are modeled in the research field of distributed systems as a Byzantine failure. Byzantine Fault Tolerance is known as a replication technology that allows the entire distributed system to continue providing a service even if some nodes are Byzantine. This technology is widely used as a core technology for blockchain (distributed ledger) and distributed databases. In this laboratory, we are conducting research on various technologies such as consensus algorithms to realize more effective Byzantine Fault Tolerance, especially for geo-wide replication utilizing public cloud services.

Machine Learning-based Network Intrusion Detection Systemy

Intrusion attacks on servers over the network are increasing every year. Network intrusion detection systems (NIDS) are widely used to detect and block such attacks. However, many systems use pattern files to detect attacks and cannot handle new types of attacks. To solve this problem, research on intrusion detection systems using machine learning is advancing. In this laboratory, we proposed a distributed processing framework called MLNIDS to realize an intrusion detection system using machine learning. We will verify and improve the practicality of this framework from various viewpoints such as detection rate, processing performance, and stability of long-term operation.

Theoretical Analysis of Cooperative Behavior by Autonomous Mobile Robots

Autonomous mobile robot attracts much attention as a new distributed system implementation. A mobile robot moves autonomously in an environment and collects information. Although each autonomous mobile robot has limited functions, multiple robots can exchange information collected individually and perform advanced tasks cooperatively. From the viewpoint of theoretical computer science, we aim to clarify the functions and complexities of mobile robots required to solve various tasks. The robot model has many variations, such as the presence or absence of memory and communication functions and the definition of fields (continuous and discrete planes).

Data Science for Materials and Drug Discovery

In chemoinformatics, an interdisciplinary field at the intersection of chemistry and information science, the primary focus is on developing technologies to accurately predict the bioactivities of drug candidates and the properties of substances. Such technologies are useful for reducing the number of experimental trials, leading to the computer-driven development of new drugs and functional materials. Our laboratory conducts data science research on molecules, including the development of machine learning models to predict various molecular properties with high accuracy, the proposal of new molecular features and descriptors for machine learning, and the construction of datasets and databases. Furthermore, through algorithm-based generation and exploration of molecular structures, we aim to design molecules exhibiting novel properties.

Development of Algorithms and Approximation Methods for Quantum Chemical Calculations

Quantum chemical calculations are numerical calculations approximately solving the Schrödinger equation and the Dirac equation, which are the fundamental equations of quantum mechanics, for molecular systems. The quantum chemical calculations enable the prediction and explanation of molecular properties and reactivities, used as tools in research and development for chemistry, materials science, and life science. However, sufficiently accurate calculations are occasionally too expensive even for today’s supercomputers. Developing new algorithms and approximation methods to improve the accuracy of quantum chemical calculations, reduce computation time, and expand their applicability will accelerate research and development in the fields of pharmaceuticals and materials.

Elucidation of Microscopic Mechanisms Underlying Molecular Photoproperties

Molecules absorb and emit electromagnetic waves, and the wavelengths of the absorption and emission vary from molecule to molecule. The photoabsorption and photoemission properties are utilized in materials such as dyes, pigments, organic light-emitting diodes (OLEDs), and solar cells. The absorption and subsequent processes occur in very short time, making it difficult to elucidate the detailed mechanisms through experimental techniques. Computational science methods such as quantum chemical calculations are well-suited for analyzing short-time phenomena at the molecular scale. Our laboratory performs large-scale numerical calculations using cluster systems and supercomputers to elucidate the mechanisms of photo-induced phenomena to obtain insights useful for materials development.

Almost all the humans use spoken dialog, which is the most natural communication method. If we can recognize/manage/ synthesize speech in computers, this speech can be not only the best method of communication but can also be used as data storage media. I am engaged in technologies on spoken language.

Speech recognition

Making transcriptions of monologues such as lectures is a very promising research area. We improve acoustic modeling of the human voice using models such as the Hidden Markov Model (HMM)　and Deep Neural Network (DNN), and statistical language modeling (N-gram) . We are also investigating DNN-based End-to-end speech recognition methods.

Friendly spoken dialog system

The first impression of a spoken dialog system for novice users is that it is unnatural, because the time-lag between a human utterance and the system reply is too long and as such the user cannot distinguish whether or not the system works. This is one of the reasons why users do not feel that spoken dialog systems can be used in a comfortable, friendly manner. Thus, we focus on prosodic features like timing and pitch change in a dialog. Our dialog system has begun to speak with appropriate prosodic features considering previous user utterances. When the dialog gets lively, the pitch of the system utterances chase the user’s pitch. On the other hand, we also study a semantic dialog strategy. We are now developing a robust and natural response generation method in a system that considers its own misunderstandings.

Multimodal interface

Human often uses gestures such as finger pointing and gaze to transmit his/her intention. We are trying to realize such interaction between human and machine. Consider the operation of an autonomous vehicle. How do you let it know where you want to go and where you want to turn? It is useful if you can use finger pointing and gaze. We are developing an autonomous vehicle with such interface!

We are interested in two research areas: natural language processing and applied information system. The first research interest is natural language processing to assist human intellectual activities and to enhance human intellectual ability. Our second research interest is how to design large-scale applied information systems such as the university educational computer system and the campus network and how to operate them. In the latter area, our advantage is that we can access the real systems and networks which are used in our university campus (the photograph of this page shows a part of the systems).

Automatic summarization and organization of lecture slides and lecture speeches

There is increasing interest against e-Learning contents, in order to provide flexible supports to students who are diverse in their understanding ability. Therefore, many educational organizations work at gathering lecture slides and recording lecture speeches to construct e-Learning contents. However, such e-Learning contents have a crucial problem: if a lecture speech has neither index nor cues, its skip listening is too difficult for e-Learning users to study the lecture efficiently. We are tackling automatic summarization and organization of lecture slides and lecture speeches, in order to resolve the above problem.

Information visualization to compose multiple information sources considering user intent

The internet is a treasury of consumer generated contents, but it is quite difficult to distinguish practical, useful, accurate content among impractical, useless, inaccurate, noisy content. In order to resolve this problem, we focus into methods to improve information reliability. The first one is textual entailment, which is a task to determine whether the meaning of the hypothesis sentence can be inferred from the meaning of the premise sentence. The second one is text reuse detection, which is a task to determine whether a given text is original or not and to discover its origin when it contains reused part. The final one is visualization of relationship between texts which are collected by the above two methods.

Improvement of availability of large-scale applied information systems and networks

The modern society widely depends on information systems and networks which run 24/7. However, it is quite difficult to realize high-available large-scale complex information systems and networks because their complexity frequently bring oversights of their designers which cause unexpected outage. In order to resolve this problem, we are investigating a method to improve automatic monitoring of information systems and networks and to prevent outages.

Natural language is indispensable for inter-human communication. Likewise, if an artifact has a good command of language, that would enhance human-machine communication. Our research group is developing computer programs that can deal with natural language for helping various human activities, including information retrieval, machine translation, and speech interface. We are also studying intersectional area of the above-mentioned research topics, including spoken document retrieval, crosslanguage information retrieval, and spoken language translation.

Information retrieval

Development of intellectual activity support technology to discover the necessary information from large-scale data. In particular, we are studying technologies that make use of spoken language for information retrieval. Firstly, we are studying spoken document retrieval, in which the target documents are not textual but speech data. Secondly, we are studying speech-driven retrieval, in which user’s information needs are expressed by using spontaneous speech. For these tasks, we have been organizing the evaluation task called SpokenDoc and SpokenQuery&Doc at the NTCIR project (http://research.nii.ac.jp/ntcir/index-en.html) to promote the research activities on those research topics.

Machine translation

Development of technology that supports communication between cultural regions that use different languages. Out of various methods proposed for machine translation, we are studying statistical machine translation that learns probabilistic models from parallel translation corpus that have the same content corresponding between languages. We also develop cross-language question answering that finds the answer from information sources written in other languages using machine translation.

Speech interface

Development of technology to have a dialog with artifacts using human spoken language. We research language models that capture the characteristics of words used for various applications such as question answering and machine translation. We are also studying para-linguistic event detection, including laughter and interest detection from speech. Recently, we are also challenging medical diagnosis code retrieval from doctor-patient conversation

Machine learning techniques are widely used for various applications such as pattern recognition and robot control. We study fundamental theories of machine learning based on statistical and information theoretic methods, and apply them to data analysis problems.

Analysis and development of statistical learning methods

Bayesian inference provides a framework for solving learning and inference problems. We aim at analyzing and devising learning and inference methods, and apply them to problems such as data analysis and visualization.

Lossy data compression and rate-distortion theory

The rate-distortion function shows the minimum code length required for reconstructing compressed data under allowed distortion levels. We aim at evaluating rate-distortion functions of distortion measures used in practical learning algorithms and information sources modelling real data generation processes.

Spoken Dialog Systems

We aim to realize natural dialog systems that allow humans and machines to enjoy the interaction itself. Our research focuses on response timing control, multimodal dialog, and speech technologies for the elderly. By integrating these components into a real-time spoken dialog system, we aim to bring natural dialog systems one step closer to practical realization.

Response Timing Control

We are also developing adaptive dialog strategies that can flexibly adjust system behavior according to the user’s individual traits and situational context. This research is geared toward practical applications in diverse fields such as welfare, education, and entertainment.

Multimodal Dialog Systems

Our goal is to build systems that can not only understand the content of spoken utterances but also interpret and generate non-verbal cues such as prosody, pauses, gaze, and facial expressions.

We aim to understand how humans perceive the environments and ourselves, behave in the natural environments, and communicate with others scientifically with psychophysical and cognitive neuroscience methods. Embodied perception is a main perspective in our research. Our perceptual process and communication are crucially connected to our body physically and psychologically. We are investigating three research themes based on this perspective.

Science for Mobile Observer

To understand perception for mobile observers, we are investigating self-motion perception, 3-D objects, scene and human-body recognition across viewpoints with psychophysical experiments. To know interaction of perception and action, we are measuring motor behavior and perception during action such as walking and driving a car. Parts of driving study are cooperative studies with motor companies. We are developing a system to experience tele-presence of walking using virtual reality.

Science for Embodied Reality

To know what is realty, we are focusing on body perception, ownership and self-agency. Perceptual reality must be based on our body and its perception. We are investigating material perception, perceptual aesthetics, lightness perception, self-motion perception, human-body perception, and augmented body in virtual-reality environments. Cross-modal studies such as vision-vestibular interaction on postural control and face-voice interaction on emotions are included in the theme. We are modifying human body appearance and function using virtual reality to see how human mind and behavior change in such modified bodies.

Science for Implicit Social Cognition

We interact with others naturally, and perceive the world and others based on social communications. The crucial factor for implicit social cognition is our body. We are investigating body perception, neurophysiology of empathy, equity, and moral. We found that the preverbal infants show sympathy for others in distress and that humans can empathize with humanoid robots.

We can see and recognize things, and act without feeling any difficulties. Our mission is to explore visual function and the mechanisms of the brain which allow us to do so, and to develop new technology, including color universal design, spectral imaging, for visual information processing based on the fundamental vision research.

Vision Science – why is it seen as it is? –

Vision is far skillful rather than we imagine. Visual perception results from “interpretations” of the retinal images. The visual system has many important features including adaptation to the visual environments, integration of various visual clues to estimate 3D information, etc. Understanding these functions may make present media technology remarkable progress. Moreover, although we are almost always subjected to a barrage of different source of visual information, our visual system does not process all the information. Rather, by so-called visual attention, the visual system selectively processes some extent of the input image. To explore the fundamental functions of the vision, we are doing psychophysical experiment, EEG measurement and analysis, and constructing mathematical models of the vision. We are now tackling various topics as: Color vision; motioncolor interaction; surface quality perception; ERP studies on face processing, visual naturalness, awareness; brain–computer interface.

Vision Technology – Media technology like human vision or beyond it –

Our life has surely become convenient since image media such as a digital camera has developed and spread rapidly. It is true, however, that capability of such image media is nothing like as good as that of the human vision because it is far easy for our vision to capture image information, code the information, and recognize it. The purpose in this project is to crystallize as “technology” the knowledge acquired by the fundamental research of vision science. We are now tackling various topics as: Color blind experience filter; visualization of invisible information using spectral imaging.

Our approach is to use non-invasive method for measuring brain such as EEG, to clarify our cognition and behavior and apply these results to brain-machine interface (BCI) and neuromarketing.

Face processing

Facial color is important information for our social communication, because it provides important clues to recognize someone’s emotion and health condition. Our previous EEG study shows that face sensitive ERP component (N170) is modulated by facial color, which suggests that face color is important for face detection (Minami et al. 2011). Moreover, facial color sensitivity of N170 was found at the left occipitotemporal site (Nakajima et al. submitted). Although the EEG study suggests that N170 at the left occipito-temporal site is related to facial color processing, it has been controversial issue which brain region is involved in facial color processing, because EEG measurement has low spatial resolution. Therefore, the present study will examine the brain regions related to facial color processing by using functional magnetic resonance imaging (fMRI) with higher spatial resolution.

Information processing and Insight

For user-kindly information technology, information should be controlled and selected depending on our brain states. For this purpose, we must know users’ understanding of information. Our previous study is to estimate whether a subject recognized an object or not through a single-trial analysis of EEG, ambiguous or clear state. Six subjects (mean age was 22 years) participated in the experiments. A pool of 120 paired pictures was used for the experimental task. One is an original color image (CI) and the other is the binarized picture (BI). A classification result revealed a good discrimination with the accuracy greater than 90% (Noritake, Minami and Nakauchi, 2009). In the next, I intend to investigate the dynamics of brainstorm: the transition process from ambiguous to clear state. In addition, we will develop applied system enabling real-time estimation of the level of understanding.

Neurocommunication

The affect misattribution procedure (AMP) was proposed as a technique to measure an implicit attitude to a prime image. In the AMP, neutral symbols (e.g., a Chinese pictograph, called the target) are presented, following an emotional stimulus (known as the prime). Participants often misattribute the positive or negative affect of the priming images to the targets in spite of receiving an instruction to ignore the primes. The AMP effect has been investigated using behavioral measures; however, it is difficult to identify when the AMP effect occurs in emotional processing?whether the effect may occur in the earlier attention allocation stage or in the later evaluation stage. In this study, we will examine the neural correlates of affect misattribution, using event-related potential (ERP) dividing the participants into two groups based on their tendency toward affect misattribution.

Humans can perform various complex and dexterous movements. Even simple motions we do mindlessly in daily life are realized by excellent ability of information processing that extracts required information for the motions from complex external information obtained through various sensory organs, and control their limbs properly to establish the task. Our focus is on perception by the integration of sensory information (e.g. object recognition), motion (e.g. reach and grasp movement and handling of object) and learning function. We aim to elucidate those excellent information processing mechanisms for cognition and motor control of the central nervous system that achieves humans’ skillful movements from the viewpoint of computational neuroscience.

Computational studies of the voluntary movement of the human limb

The hand and arm trajectory of various movements such as the reach and grasp movements, handwriting movements, drawing line or figure and sports movement under a various conditions is measured by a three-dimensional motion capture system, and at the same time, other biological signals such as eye movements or electromyography signals are also measured. With the measurements, we analyze the achieved motion trajectory, change in trajectory accompanied by motor learning and cognitive information. Based on the results, we work to elucidate the principles of motion planning, motor control system and learning function by developing mathematical models of the information processing of motion control using neural models that can reproduce the measured human movements with computer simulation.

Applied research of the model for motor control

We apply the features and measurement technology of human movement obtained by motion analysis to develop a user-friendly man-machine interface, welfare technology, and robotics. For example, we investigate the sign-language translation system using the feature of human arm movements and the back-parking assist system using the motor learning theory. These applied researches also aim to confirm the validity of the model of information processing regarding motor control or learning that we constructed.

Although humans and animals have great information processing functions, many of such features are yet to be elucidated. Thus, we are studying ways to elucidate the information processing process of those functions and consider a breakthrough based on what we learned from biological information processing when we face an obstacle or difficult problem in artificial information processing. A wide range of academic disciplines must be taken into consideration to examine the information processing mechanism of humans and animals. Therefore, we thoroughly examine physiological and psychological knowledge and proceed with our studies from a multidisciplinary view while taking an information scientific approach. The ultimate goal is to create an artifact with functions superior to that of humans and animals.

Intelligent information processing

We aim to realize the intelligent information processing that humans can perform well and suggest soft computing techniques of neural circuit models, flexible reinforcement learning and self-organizing maps etc.

Sensory information processing model

We explore the mechanism of perception information processing that humans can perform well by applying the modeled visual processing that humans can do well to image processing and modeling the visual process with consideration to the mechanism of processing illusions.

We explore how people feel and think, and how these processes give rise to behavior, thereby deepening our understanding of the human mind.

Why are our behaviors different?

Even when we experience the same event, our experiences can vary across individuals. For example, people differ in their tastes in music. We aim to elucidate the mechanisms underlying such individual differences.

Why is our memory fragile?

Human memory is not always accurate; we may even recall events as if they had actually happened, despite never experiencing them. We study why memory is so fragile and what this reveals about human nature.

How are the body and mind connected?

We reason about the world around us and draw logical conclusions. These processes are often assumed to be independent of the body, but is that really the case? We investigate how the body shapes human reasoning and what this reveals about human cognition.

Our laboratory conducts neuroscience research to elucidate the neural substrates underlying neural information processing for sensorimotor and cognitive control, as well as flexible decision-making. Using our empirical findings, we develop apparatuses to aid in physical education and rehabilitation

Cognitive & motor control

We conduct studies to comprehend the key components including neural dynamics that contribute to cognitive and motor control. e.g., Speed-accuracy tradeoff, perceptual decision making, and predictive coding. We conduct an EEG-TMS approach and neurofeedback to assess the causal relationship between brain activity and behavior.

Learning & Motor skill

We focus on comprehending the neural substrates underlying cognitive and motor learning, with a specific emphasis on the relationships between neural dynamics and excitatory-inhibitory balance.

Artificial intelligence

In recent years, AI has been adopted for multiple biological data analysis such as diagnosis of pathologic conditions. However, the trustworthiness, robustness, and transparency of AI are still debatable. Utilizing neuroinformatics technologies such as neural activity manipulation and virtual biological signal manipulation, we aim to verify the trustworthy AI systems when AI predicts our behavior using biological signals (EMG and EEG).

We address the issues related to auditory perception and cognition using various psychological experiment paradigms, mainly psychophysical methods. Our research scope covers various themes from computational modeling of the early stage of hearing to cognition of music. We aim to reveal the full function of hearing by approaching from both low-level processing and that of higher function.

Computational model of auditory system

In the auditory system there still remains some unexplained functions; the auditory path from the outer-ear to the auditory area of cerebral cortex is too deep inside to observe from the outside and the path is complicated due to lots of nuclei relays. To understand such hearing has been recently facilitated by computational models that express the processing at each stage of hearing by signal processing. We measure the fundamental functions of hearing such as encoding frequency components, period of waveform, and dynamic range of sounds by psychophysical experimental methods, and reflect the results in a computational model called dynamic compressive gammachirp filterbank model which has been developed in collaborative research projects. By following the prediction of perceptual phenomena by the computational model and its experimental validation, we contribute to the understanding of the human auditory system.

Hearing impairment simulation and its application

Japan is experiencing a “super-aging” society nowadays. It is predicted that age-related hearing loss will increase with the increase in the population of elderly people. Although it is necessary to avoid disconnection of communication due to hearing loss, it is so hard to imagine how hearing loss changes the perception of sound. The computational model of the auditory system can output not only an expression of auditory function, but also sounds deteriorated by modules causing hearing loss. It allows users “to listen as a person with hearing loss”. We are planning to apply this hearing impairment simulation to an educational course for speech therapists, and disperse broadly to the general public to learn about hearing. This simulation will also allow us to evaluate sounds with specific hearing loss, and obtain cues to synthesize clear sounds for all of us.

Changes in perception and cognition by long-term training of music

The questions of how we perceive music as an object with recognizable temporal structure, receive various emotions from it, and enjoy it are still research themes stimulating many researchers. We will study how music changes human audio-visual information processing with the cooperation of professional musicians intensely trained since childhood. We are also investigating the diversity of musicians who have often been treated as a single group

We are developing novel technologies based on artificial intelligence (AI) such as machine learning and neural network, for smartly generating, editing, and analyzing visual contents. Also, mathematical models or applications are developed to create contents or values in a real world, such as digitally designed craftworks or lighting environments, from graphical or image representation.

Humanoid animations based on motion data learning

Classification, recognition, and conversion of human motion data has a big potential for developing digital human technologies in various fields: gestural or behavioral recognitions, simulations for physical trainings, and humanoid animations for video games and VR systems. This project introduces advanced numerical optimizations or machine learning techniques such as deep neural networks to motion data generation or recognition.

Style-based graphics and image processing

Machine learning techniques are rapidly progressed in the research fields of graphics and image synthesis. We are interested in applying state-of-the-art deep learning algorithms to these fields, especially in style manipulation/conversion of graphical/image representations such as motion capture data and illustrative or cartoon-like images.

Image-based craftwork and optical control

This project develops the methodologies or novel applications based on artificial or illustrative images. Smart image processing such as conversion, sampling, style recognition, is developed to support the designs and creations of physical object, such as machine embroidery and smart controls of decorative illuminations, using state-of-the-art methods of machine learnings.

We are developing novel technologies based on artificial intelligence (AI) such as machine learning and neural network, for smartly generating, editing, and analyzing visual contents. Also, mathematical models or applications are developed to create contents or values in a real world, such as digitally designed craftworks or lighting environments, from graphical or image representation.

Humanoid animations based on motion data learning

We aim to develop intelligent systems, such as service robots and self-driving vehicles, which can operate autonomously and intelligently in complex real environments. A key to realize such systems is advanced information processing or AI, including scene recognition, context-aware planning, and human-machine interaction.

Attendant robot / Service robot

Autonomous service robots are expected to support our daily life in various scenarios, such as attending, guiding, errand, and search. We have been developing novel methods for such service robots, including robust scene recognition, human detection/tracking/identification and pose estimation, and task and action planning.

Outdoor navigation / Autonomous driving

Outdoor navigation is one of the necessary functions of service robots since outdoor environments are important parts of human activity. We have been developing novel methods for outdoor navigation such as multi-sensory road boundary tracking, view-based robust localization, and large-scale 3D mapping and localization. Many of these methods can be applied to autonomous driving.

Human-robot interaction

Human-robot collaboration is an effective way of achieving non-trivial tasks in complex real environments. We have been developing novel approaches in various collaborating scenarios such as human-robot collaborative assembly, collaborative remote object search, and robot-mediated task knowledge transfer.

The Information and Robot Systems Laboratory (Kakiuchi Laboratory) studies and develops robot systems that contribute to realizing a more prosperous society.

By focusing on robot systems, we are conducting research and development in various fields, including hardware, software, and systemization, to explore fields where robots can be used effectively, new ways of using robots, and new ways of making robots.

Humanoid Robot System Integrating Environmental Awareness and Motion

We are studying robot systems that enable humanoid robots to operate autonomously in real environments.
We not only perform dynamic calculations for the robot using simulations, but also simulate sensors such as cameras, and build the same system as the actual robot on the simulation to verify the system and construct the robot’s recognition behavior.

By repeating the verification of the behavior performed while in contact with the environment using simulations and actual machine operation, we aim to build a robot system that can operate continuously in real environments.

Adaptive Robot Creation System

To increase the number of problems that robots can solve, we will study a system for creating robots suited to the desired task.

Conventional robot systems are designed for a specific purpose, and the necessary systems are constructed and provided, but we are studying a system that can propose a robot system that solves a problem.

We build a system that can easily construct a robot that combines modular actuators and can verify its operation, and by updating the robot configuration based on the operation verification, we aim to create an adaptive system. We aim to create a system that can automatically propose a robot system configuration by accumulating data from designers who have designed robots suitable for a specific problem.

The “ubiquitous” refers to an environment in which various “things” around us have information processing capabilities and support our lives. In the Ubiquitous Computing Laboratory, we are conducting research from various perspectives, including new services, hardware devices for realizing those services, system software, networks, cloud services, machine learning algorithms, smartphone and web app development, etc., with the aim of creating a world in which computers actively and autonomously support our activities by connecting the real world and the virtual world.

Wearable Computing x AI Data Analysis

We are studying sensor devices that are worn on the human body and technologies that use AI to process the sensor data obtained from these devices to estimate the content of a person’s activities. We envisage that the estimated activities will be used to ensure the safety of the work environment, reduce the workload, and improve services for nurses and childcare workers.

IoT x Smart City x AI Analysis

We are developing IoT devices to observe indoor and outdoor events.We are also conducting AI analysis of the acquired data to study how to revitalize towns and improve safety and convenience.

Mobile Network Applications x Smart Cities

We develop applications to support local tourism and traditional events.

The aim of computer vision is to realize the functions of human vision on a computer and its applications are widely used for various fields: 3-D measurement, surveillance system, robot vision, medical image processing, and so on. In this laboratory, we focus on 3-D reconstruction from images and matching between images. We also study image processing for colorblind persons and a safety system for vulnerable road users like children and elderly persons using image processing.

Accurate image matching

Image matching is the first step for many computer vision applications like 3-D reconstruction and object recognition. The accuracy affects to that of reconstructed 3-D shape. We are studying robust and accurate image matching methods for various scenes.

3-D reconstruction from images under

special environments Endoscope image sequence is one of difficult case for accurate 3-D reconstruction because of the special motion of the camera. Drone images of the sands or beach is also difficult to obtain accurate 3-D shape because the target is almost planar and its texture is almost the same. We are conducting the research of accurate 3-D reconstruction under such special environments.

Image enhancement for colorblind

person Colorblind person feels inconvenience in daily life because the color design of almost things is not adequate for them, e.g., traffic signs/signals, road/ floor maps, even web pages. We have proposed image enhancing method by additive image noise for them. By our method, not only colorblind person can distinguish their indistinguishable colors but also normal person can recognize the original colors. We are conducting the improvement of our method.

We are studying geometric problems of computer vision including 3-D reconstruction, ellipse fitting, ellipse detection, and mixed reality system.

Mixed reality system using circular marker

We are studying ellipse detection from images and use this technique for detecting a circular marker of a mixed reality system. The advantage of using a circular marker is that we can estimate a camera position and pose from this marker if it is hidden by other objects.

Developing diminished reality system

We are developing a diminished reality system using circular markers and a 3-D shape of a background scene obtained from a RGB-D sensor like the Kinect. We are now focusing an alignment between the overlaid background and a captured image from a user camera.

Ellipse fitting and ellipse detection

We are studying a high-accuracy ellipse fitting technique and ellipse detection from images for applying mixed reality system and camera calibration.