We are engaged in research and development on information security and cryptographic technologies that are required for secure communication and information processing on the Internet. Modern cryptographic technology is designed to be provably secure based on computational hard problems such as prime factorization problem and discrete logarithm problem, so theoretical analysis to prove its security is required. On the other hand, cryptographic technology is actually used in the Internet, so efficiency and practicality are also important. Based on these theoretical and practical viewpoints, we do research and development on information security, especially public key cryptography and cryptographic protocols, with the following research topics.

Cryptographic Protocol

We do research and development on cryptographic protocols, which realize advanced functions using cryptographic primitives, e.g., key exchange protocol TLS widely used in the Internet, and distributed ledger technology block chain used in Bitcoin etc. We design and implement secure and efficient cryptographic protocols.

Functional Encryption

We do research and development on functional encryption and other highly functional cryptographic primitives, e.g., ID-based encryption, and anonymous signature etc. We also study and implement efficient algorithms for elliptic curve and pairing, which is the basis of the functional encryption and highly functional cryptographic primitives.

2-party / Multi-party Computation

We do research and development on 2-party / multi-party computation, which is the key technology to utilize sensitive information, e.g., personal information and medical information, effectively, while keeping privacy. 2-party / multi-party computation is a “universal” cryptographic protocol that can compute arbitrary function on inputs of the parties, without revealing the inputs.

A wide range of computational tasks are in great demand in modern society and industries. Dealing with them without algorithmic ingenuity, however, often causes an exponential explosion in processing time, or otherwise, results in poor outputs. It is thus said that most crucial and essential portion of softwaredevelopment in such cases consists of algorithm design. Therefore, we aim to develop highly efficient and accurate algorithms for various problems by making use of discrete structure analysis, algorithm theory, computational complexity theory, mathematical programming, among others.

Design and development of new algorithms and models

Mainly focusing on important combinatorial optimization problems (and typically abstracting them into problems of graphs, networks, and sets) appearing in the areas of production / delivery planning, scheduling, VLSI design, optimal routing, and others, we devise and design new algorithms and mathematical models for them. Various types of algorithms are of our interest such as approximation, distributed/parallel, online/streaming, etc.In order to verify the validity and effectiveness of the algorithms and models thus designed, their performance will be evaluated by theoretical analysis and / or computational experiments.

New ways of designing optimization algorithms

Studied here are principles and mechanism of optimization that can serve as driving forces for algorithms solving combinatorial optimization problems . Compared to Theme 1, such study requires consideration from a more meta viewpoint.Examples under this theme include design techniques based on the duality theorem and complementary slackness conditions from the area of linear programming.

Study on the constraint satisfaction problem (CSP)

CSP is recognized as one of the most fundamental problems in computer science with applications in diverse fields such as artificial intelligence, operations research, and otheres. Recently, it is shown that CSP with a finite domain has the dichotomy property . We study the complexity of variants of CSP.

Online / Stream optimization

Online algorithms process input data given in sequence along the time – series(implying that input coming in the future is unpredictable).It has applications in a wide range of areas such as power -saving control, logistics, and financial engineering. In the large-scale data processing of present days, it is also crucial to to process input sequence data without (or with strictly restricted) need for data storage, using streamingalgorithms

In the Course of Study for high school that went into effect in 2003, the new subject “Information” was added as a compulsory subject. It made all high school students receive information education. In addition, the “Information and Computer” unit of technology and home economics in junior high school became compulsory. Elementary schools have classes and information education using computers at “Period for Integrated Studies” and classes for other subjects. The importance of information education is commonly recognized in school education at every level. Educational efforts to develop the ability to utilize information, which is the target of information education, have been made beyond the framework of subjects. I am among those who are making such effort. I work with teachers of local elementary, junior high and high schools and conduct practical research on how the new way of information education should be. Main research themes include the following.

Information education at primary and secondary education

The figure below shows the book educational support system for the school library. Part of this system has been introduced in all elementary and junior high schools in Toyohashi city.

Computer use in language education

Science cafe for lifelong education in community

I believe the most important thing for students is to have them decide the theme for their graduate research and master’s thesis themselves when I provide students guidance. Choosing a research theme is the hardest yet most interesting thing. It is the best part of the research. As an educator, I bear in mind not make the idiotic mistake of assigning research themes to students.

We have been studying on the electronic properties of biological macromolecules such as DNA, proteins and ligand using ab initio molecular simulations, in order to propose novel potent medicines for treating globally feared diseases such as Alzheimer’s disease, tuberculosis and cancer. In addition, to elucidate the transcription mechanism of genetic information from DNA to RNA, which is controlled efficiently by some transcriptional regulation proteins, we investigate the specific interactions between DNA and these proteins by the ab initio molecular simulations. Some of the key research themes of our laboratory are shown below.

Elucidation of aggregation mechanism of amyloid-beta (Aβ) proteins

Onset of Alzheimer’s disease is shown to be deeply associated with the formation of amyloid plaque in a patient’s brain. However, the formation mechanism of the plaque from Aβ proteins is not clarified yet. We are in an attempt to elucidate the mechanism, in order to propose novel compounds for preventing the formation of the plaque. These compounds might be potent inhibitors against Alzheimer’s disease.

Proposal for potent medicines against Alzheimer’s disease

We investigated the specific interactions between Aβ protein and many kinds of compounds and proposed novel inhibitors against the aggregation of Aβ s. These inhibitors are the derivatives of curcumin, which is included in natural product turmeric. Our collaborators in Ukraine are attempting to synthesize these derivatives, and they are expected to be potent medicines for Alzheimer’s diseases.

Proposal for potent medicines against tuberculosis

Tuberculosis (TB) is one of the most widespread infection diseases in the world, and many types of anti-TB drugs were developed. However, since Mycobacterium tuberculosis (Mtb) can easily get drug resistance, it is necessary to develop novel effective anti-TB drugs targeting the most conservative proteins. We employed the filamental temperature-sensitive Z (FtsZ) as the target protein and investigated the binding properties between FtsZ and many types of compounds using ab initio molecular simulations. Based on the results simulated, we proposed some novel compound inhibiting the polymerization of FtsZ proteins, which causes the cell division of Mtb.

Proposal for potent inhibitors against cancer metastasis

To develop more potent inhibitors against the cancer metastasis, we investigated the binding properties of amino acid peptides with a protein, which is considered to be a trigger for the onset of cancer metastasis. Based on the results simulated, some novel peptides having large binding affinity to the protein were proposed as potent inhibitors. Our proposed new peptides are expected to be new inhibitors against cancer metastasis.

Elucidation of the transcription mechanism controlled

by regulatory proteins The transcription mechanism of gene information from DNA to RNA is efficiently controlled by various regulatory proteins as well as ligand molecules. The catabolite activator protein (CAP) is one of the regulatory proteins, and the complex of CAP with cyclic AMP (cAMP) plays an efficient role in the transcription mechanism. We investigated the effect of cAMP-binding to CAP on the interactions between DNA and CAP using ab initio molecular simulations, in order to elucidate the transcription mechanism controlled by CAP and cAMP. The results will be helpful for proposing novel compounds controlling the transcription mechanism.

The Computer Systems Performance Engineering Laboratory studies performance of any platforms of computers from ultra high-speed supercomputers to extreme low-power mobile devices based on scientific and engineering approaches. Currently, many deep learning programs, which become a typical component for AI-based applications, has struggled with lack of performance even if it is implemented on the state-ofthe-art CPUs or GPUs. Therefore, techniques that realize high-performance and high-efficiency computer systems are expected to be an enabler of emerging new AI applications such as self-driving cars and autonomous intelligent robots. In the context of system performance matter, we highlight specialization for inherent memory access locality which will be a clue to its solution.

Memory-centric customization toward FPGA accelerators

For realizing a human-centered super smart society called Society 5.0, it is very important to attain highperformance and high-efficiency computing through custom computing systems specialized for particular application domains. On the other hand, customization of systems incurs challenges in productivity and costs for the hardware and software development process. To overcome these difficulty, we investigate the ways to detect locality of processing inherent in application programs and develop high-level optimization frameworks that automatically map it to FPGA accelerators.

Scientific modeling of system performance and quality of the performance

We investigate the ways to profile, analyze and predict behaviors of program execution to understand the software execution performance. Here, we develop profilers, performance models, and simulators of hierarchical memories.

Automation of customization and co-design driven by mathematical optimization and

machine learning techniques We investigate the ways to automate co-design processes for domain-specific customization. These are implemented upon the current software stack such as compilers, code translators, and optimizers. These are also strongly tackled with mathematical optimization, machine learning, and deep learning techniques.

My laboratory develops new methodologies and software applications on computational chemistry and chemoinformatics using the latest computational science and information technologies, such as highperformance parallel distributed programing, data assimilation using computer simulation and database, and machine learning with deep neural net technology. The main research themes are introduced as follows.

Exploring Molecular Conformation and Crystal Structure Polymorphism

Many of drug molecules and organic molecular materials are used as solid crystals, and their medicinal effects and material properties are determined by their stable crystal structure. However, in these molecular crystals, it is possible that a molecule in crystal may form different 3D structures (conformations) or different crystal structures (polymorphisms). Therefore, prediction of some stable crystal structures in advance is important information for designing pharmaceutical molecules and organic materials. Our computational chemistry application CONFLEX that can be exploring molecular conformations and crystal structure polymorphisms, is developed as a supporting tool for advanced molecular design.

Protein-Ligand Docking Simulation by using Coarse-Grained Potentials

A drug molecule binds to a target protein and forms a complex structure. As a result, information transmission in vivo is activated or inhibited, and the function expression of the protein is controlled. In order to elucidate essential biomolecular behaviors and biological phenomena in the fields of drug discovery, we are developing a new docking simulation method so that can efficiently search some plausible ligand binding sites by a coarse-grinded model and rapidly produce highly accurate all-atom model by data assimilation with conformation database.

Molecular Activity and Material Property Prediction by using Deep Neural Nets

Machine learning technology using deep neural nets has made amazing results in the field of speech recognition and image analysis. Furthermore, it is expected that paradigm shift will occur in structural activity correlation of drug molecules and prediction of physical-chemical properties for electronic device materials. Our laboratory formulates new molecular descriptors in terms of structures, energies, and electronic states based on computational chemistry simulation, and develops a next generation system on molecular activity and property prediction by using some deep learning platforms.

Many operational setups can be conveniently described by a circuital model. Whenever an input is provided, the circuit converts it into some sort of information, that is then processed according to some rules and finally converted back to produce an output. However, classical information theory does not suffice for an adequate description of the way such a circuit internally represents and processes information, and one has to resort to its quantum counterpart. Our research spans such a theory of quantum information.

Quantum communication and information retrieval

Our research topics include quantum communication and information-retrieval problems, such as the minimization of the error or the energy in the readout of quantum bits. We study both the single shot case (quantum hypothesis testing, utility of quantum channels, guesswork of quantum ensembles, quantum reading of unitary devices), as well as the asymptotic regime (quantum channel capacities, accessible information of quantum ensembles, informational power of quantum measurements).

Quatum circuit characterization

We investigate the problem of characterizing quantum circuits with machine-learning-like approaches solely based on input-output correlations, such as device-independent tests and data-driven inference, with applications for instance in the reliable calibration of quantum computers. In conjunction with the derivation of simulability criteria for quantum circuits, such a characterization aims at a device-independent formulation of the resource theories of quantum  circuits.

Higher order quantum circuits

Our research lines include quantum combs, that is, quantum circuits that have other circuits as their input and output. For instance, we investigate the problem of the controllability of quantum circuits, that is, the quantum equivalent of the ”if-then” conditional construct of classical programming languages, for the implementation of algorithms for quantum computation.

We have conducted research on massive multimedia datasets including texts, images, videos, and 3D shape modes, and extract valuable pieces of information. Main focus of our research has been on feature extraction, information retrieval, clustering, classification, segmentation, and automatic annotation or tagging to multimedia.

Research on Multimedia Retrieval, Classification, and Automatic Annotation using Finegrained Feature Extraction with Deep Learning

In this research, we use three-dimensional shape models, images, videos, and annotated texts, in order to investigate fine-grained accurate search, including partial-matching and classification. The basic idea is to extract salient features that represent an object robustly and invariantly under translation, rotation, scaling, and other similar operations. We boast of the world-top-level search performance for 3D shape models, proven by SHREC international contests. We also boast of the world-top-level classification for plane identification among massive datasets, proven by PlantCLEF international contests. To keep our research up to the state-of-the-art we are currently embarking on various applications on deep learning.

Web Mining, Data Mining, Text Mining, and More

Web is considered a rich resource including a billion of stones and a handful of gems. Web mining is an emerging research field, attempting to find “gems” on the Web. Our research on Web mining includes Web content mining (such as SNS data mining), Web spam detection, Web link analysis, and Web usability monitoring. In addition we are conducting research on times series data mining, blog/microblog analysis, sentiment analysis, personalized information extraction, and prediction of trends not easily observed by ordinary users’ perspectives. We have also started research on intent mining and diversification, focusing on “what’s new” (novelty) to disambiguate multiply interpreted queries.

We are applying statistical methodology to make the best use of computer networks and the data in the works as follows.

Handling Large Scale Data from Social Media

We have been operating data collecting systems, especially from social media (Twitter), where various information is posted. As the result, we can handle almost all tweets with location information and Japanese/English retweets, and large scale social graphs. We are conducting researches to make the most of these data. One example of these researches is estimating the location of tweets and users without location information. Moreover, we are analyzing the differences between the real world and the internet world regarding human/user relations for computational social science.

Natural Language Processing for Information Retrieval

It is crucial for information retrieval systems to decide whether each term is important for retrieval. We have developed a method to extract important terms from documents using statistical analysis. Unlike commonly used approaches, this method does not require dictionaries, but collections of data. This method is the result of joint research with a company, and used in commercial products. You can get more details on the following page. http://www.sei-info.co.jp/quicksolution/technology/birth-story2.html

E-learning System in Network Era

We believe that computer technology can provide more effective contents for E-learning than simple video. One of our results is an image processing method to remove the image of the lecturer and to provide a clear image of the black board. This kind of content makes note taking easier because the lecturer tends to cover the characters on the white board.

Almost all the humans use spoken dialog, which is the most natural communication method. If we can recognize/manage/ synthesize speech in computers, this speech can be not only the best method of communication but can also be used as data storage media. I am engaged in technologies on spoken language.

Speech recognition

Making transcriptions of monologues such as lectures is a very promising research area. We improve acoustic modeling of the human voice using models such as the Hidden Markov Model (HMM)　and Deep Neural Network (DNN), and statistical language modeling (N-gram) . We are also investigating DNN-based End-to-end speech recognition methods.

Friendly spoken dialog system

The first impression of a spoken dialog system for novice users is that it is unnatural, because the time-lag between a human utterance and the system reply is too long and as such the user cannot distinguish whether or not the system works. This is one of the reasons why users do not feel that spoken dialog systems can be used in a comfortable, friendly manner. Thus, we focus on prosodic features like timing and pitch change in a dialog. Our dialog system has begun to speak with appropriate prosodic features considering previous user utterances. When the dialog gets lively, the pitch of the system utterances chase the user’s pitch. On the other hand, we also study a semantic dialog strategy. We are now developing a robust and natural response generation method in a system that considers its own misunderstandings.

Multimodal interface

Human often uses gestures such as finger pointing and gaze to transmit his/her intention. We are trying to realize such interaction between human and machine. Consider the operation of an autonomous vehicle. How do you let it know where you want to go and where you want to turn? It is useful if you can use finger pointing and gaze. We are developing an autonomous vehicle with such interface!

Natural language is indispensable for inter-human communication. Likewise, if an artifact has a good command of language, that would enhance human-machine communication. Our research group is developing computer programs that can deal with natural language for helping various human activities, including information retrieval, machine translation, and speech interface. We are also studying intersectional area of the above-mentioned research topics, including spoken document retrieval, crosslanguage information retrieval, and spoken language translation.

Information retrieval

Development of intellectual activity support technology to discover the necessary information from large-scale data. In particular, we are studying technologies that make use of spoken language for information retrieval. Firstly, we are studying spoken document retrieval, in which the target documents are not textual but speech data. Secondly, we are studying speech-driven retrieval, in which user’s information needs are expressed by using spontaneous speech. For these tasks, we have been organizing the evaluation task called SpokenDoc and SpokenQuery&Doc at the NTCIR project (http://research.nii.ac.jp/ntcir/index-en.html) to promote the research activities on those research topics.

Machine translation

Development of technology that supports communication between cultural regions that use different languages. Out of various methods proposed for machine translation, we are studying statistical machine translation that learns probabilistic models from parallel translation corpus that have the same content corresponding between languages. We also develop cross-language question answering that finds the answer from information sources written in other languages using machine translation.

Speech interface

Development of technology to have a dialog with artifacts using human spoken language. We research language models that capture the characteristics of words used for various applications such as question answering and machine translation. We are also studying para-linguistic event detection, including laughter and interest detection from speech. Recently, we are also challenging medical diagnosis code retrieval from doctor-patient conversation

Machine learning techniques are widely used for various applications such as pattern recognition and robot control. We study fundamental theories of machine learning based on statistical and information theoretic methods, and apply them to data analysis problems.

Analysis and development of statistical learning methods

Bayesian inference provides a framework for solving learning and inference problems. We aim at analyzing and devising learning and inference methods, and apply them to problems such as data analysis and visualization.

Lossy data compression and rate-distortion theory

The rate-distortion function shows the minimum code length required for reconstructing compressed data under allowed distortion levels. We aim at evaluating rate-distortion functions of distortion measures used in practical learning algorithms and information sources modelling real data generation processes.

We are interested in two research areas: natural language processing and applied information system. The first research interest is natural language processing to assist human intellectual activities and to enhance human intellectual ability. Our second research interest is how to design large-scale applied information systems such as the university educational computer system and the campus network and how to operate them. In the latter area, our advantage is that we can access the real systems and networks which are used in our university campus (the photograph of this page shows a part of the systems).

Automatic summarization and organization of lecture slides and lecture speeches

There is increasing interest against e-Learning contents, in order to provide flexible supports to students who are diverse in their understanding ability. Therefore, many educational organizations work at gathering lecture slides and recording lecture speeches to construct e-Learning contents. However, such e-Learning contents have a crucial problem: if a lecture speech has neither index nor cues, its skip listening is too difficult for e-Learning users to study the lecture efficiently. We are tackling automatic summarization and organization of lecture slides and lecture speeches, in order to resolve the above problem.

Information visualization to compose multiple information sources considering user intent

The internet is a treasury of consumer generated contents, but it is quite difficult to distinguish practical, useful, accurate content among impractical, useless, inaccurate, noisy content. In order to resolve this problem, we focus into methods to improve information reliability. The first one is textual entailment, which is a task to determine whether the meaning of the hypothesis sentence can be inferred from the meaning of the premise sentence. The second one is text reuse detection, which is a task to determine whether a given text is original or not and to discover its origin when it contains reused part. The final one is visualization of relationship between texts which are collected by the above two methods.

Improvement of availability of large-scale applied information systems and networks

The modern society widely depends on information systems and networks which run 24/7. However, it is quite difficult to realize high-available large-scale complex information systems and networks because their complexity frequently bring oversights of their designers which cause unexpected outage. In order to resolve this problem, we are investigating a method to improve automatic monitoring of information systems and networks and to prevent outages.

We aim to understand how humans perceive the environments and ourselves, behave in the natural environments, and communicate with others scientifically with psychophysical and cognitive neuroscience methods. Embodied perception is a main perspective in our research. Our perceptual process and communication are crucially connected to our body physically and psychologically. We are investigating three research themes based on this perspective.

Science for Mobile Observer

To understand perception for mobile observers, we are investigating self-motion perception, 3-D objects, scene and human-body recognition across viewpoints with psychophysical experiments. To know interaction of perception and action, we are measuring motor behavior and perception during action such as walking and driving a car. Parts of driving study are cooperative studies with motor companies. We are developing a system to experience tele-presence of walking using virtual reality.

Science for Embodied Reality

To know what is realty, we are focusing on body perception, ownership and self-agency. Perceptual reality must be based on our body and its perception. We are investigating material perception, perceptual aesthetics, lightness perception, self-motion perception, human-body perception, and augmented body in virtual-reality environments. Cross-modal studies such as vision-vestibular interaction on postural control and face-voice interaction on emotions are included in the theme. We are modifying human body appearance and function using virtual reality to see how human mind and behavior change in such modified bodies.

Science for Implicit Social Cognition

We interact with others naturally, and perceive the world and others based on social communications. The crucial factor for implicit social cognition is our body. We are investigating body perception, neurophysiology of empathy, equity, and moral. We found that the preverbal infants show sympathy for others in distress and that humans can empathize with humanoid robots.

We can see and recognize things, and act without feeling any difficulties. Our mission is to explore visual function and the mechanisms of the brain which allow us to do so, and to develop new technology, including color universal design, spectral imaging, for visual information processing based on the fundamental vision research.

Vision Science – why is it seen as it is? –

Vision is far skillful rather than we imagine. Visual perception results from “interpretations” of the retinal images. The visual system has many important features including adaptation to the visual environments, integration of various visual clues to estimate 3D information, etc. Understanding these functions may make present media technology remarkable progress. Moreover, although we are almost always subjected to a barrage of different source of visual information, our visual system does not process all the information. Rather, by so-called visual attention, the visual system selectively processes some extent of the input image. To explore the fundamental functions of the vision, we are doing psychophysical experiment, EEG measurement and analysis, and constructing mathematical models of the vision. We are now tackling various topics as: Color vision; motioncolor interaction; surface quality perception; ERP studies on face processing, visual naturalness, awareness; brain–computer interface.

Vision Technology – Media technology like human vision or beyond it –

Our life has surely become convenient since image media such as a digital camera has developed and spread rapidly. It is true, however, that capability of such image media is nothing like as good as that of the human vision because it is far easy for our vision to capture image information, code the information, and recognize it. The purpose in this project is to crystallize as “technology” the knowledge acquired by the fundamental research of vision science. We are now tackling various topics as: Color blind experience filter; visualization of invisible information using spectral imaging.

Our approach is to use non-invasive method for measuring brain such as EEG, to clarify our cognition and behavior and apply these results to brain-machine interface (BCI) and neuromarketing.

Face processing

Facial color is important information for our social communication, because it provides important clues to recognize someone’s emotion and health condition. Our previous EEG study shows that face sensitive ERP component (N170) is modulated by facial color, which suggests that face color is important for face detection (Minami et al. 2011). Moreover, facial color sensitivity of N170 was found at the left occipitotemporal site (Nakajima et al. submitted). Although the EEG study suggests that N170 at the left occipito-temporal site is related to facial color processing, it has been controversial issue which brain region is involved in facial color processing, because EEG measurement has low spatial resolution. Therefore, the present study will examine the brain regions related to facial color processing by using functional magnetic resonance imaging (fMRI) with higher spatial resolution.

Information processing and Insight

For user-kindly information technology, information should be controlled and selected depending on our brain states. For this purpose, we must know users’ understanding of information. Our previous study is to estimate whether a subject recognized an object or not through a single-trial analysis of EEG, ambiguous or clear state. Six subjects (mean age was 22 years) participated in the experiments. A pool of 120 paired pictures was used for the experimental task. One is an original color image (CI) and the other is the binarized picture (BI). A classification result revealed a good discrimination with the accuracy greater than 90% (Noritake, Minami and Nakauchi, 2009). In the next, I intend to investigate the dynamics of brainstorm: the transition process from ambiguous to clear state. In addition, we will develop applied system enabling real-time estimation of the level of understanding.

Neurocommunication

The affect misattribution procedure (AMP) was proposed as a technique to measure an implicit attitude to a prime image. In the AMP, neutral symbols (e.g., a Chinese pictograph, called the target) are presented, following an emotional stimulus (known as the prime). Participants often misattribute the positive or negative affect of the priming images to the targets in spite of receiving an instruction to ignore the primes. The AMP effect has been investigated using behavioral measures; however, it is difficult to identify when the AMP effect occurs in emotional processing?whether the effect may occur in the earlier attention allocation stage or in the later evaluation stage. In this study, we will examine the neural correlates of affect misattribution, using event-related potential (ERP) dividing the participants into two groups based on their tendency toward affect misattribution.

Humans can perform various complex and dexterous movements. Even simple motions we do mindlessly in daily life are realized by excellent ability of information processing that extracts required information for the motions from complex external information obtained through various sensory organs, and control their limbs properly to establish the task. Our focus is on perception by the integration of sensory information (e.g. object recognition), motion (e.g. reach and grasp movement and handling of object) and learning function. We aim to elucidate those excellent information processing mechanisms for cognition and motor control of the central nervous system that achieves humans’ skillful movements from the viewpoint of computational neuroscience.

Computational studies of the voluntary movement of the human limb

The hand and arm trajectory of various movements such as the reach and grasp movements, handwriting movements, drawing line or figure and sports movement under a various conditions is measured by a three-dimensional motion capture system, and at the same time, other biological signals such as eye movements or electromyography signals are also measured. With the measurements, we analyze the achieved motion trajectory, change in trajectory accompanied by motor learning and cognitive information. Based on the results, we work to elucidate the principles of motion planning, motor control system and learning function by developing mathematical models of the information processing of motion control using neural models that can reproduce the measured human movements with computer simulation.

Applied research of the model for motor control

We apply the features and measurement technology of human movement obtained by motion analysis to develop a user-friendly man-machine interface, welfare technology, and robotics. For example, we investigate the sign-language translation system using the feature of human arm movements and the back-parking assist system using the motor learning theory. These applied researches also aim to confirm the validity of the model of information processing regarding motor control or learning that we constructed.

We address the issues related to auditory perception and cognition using various psychological experiment paradigms, mainly psychophysical methods. Our research scope covers various themes from computational modeling of the early stage of hearing to cognition of music. We aim to reveal the full function of hearing by approaching from both low-level processing and that of higher function.

Computational model of auditory system

In the auditory system there still remains some unexplained functions; the auditory path from the outer-ear to the auditory area of cerebral cortex is too deep inside to observe from the outside and the path is complicated due to lots of nuclei relays. To understand such hearing has been recently facilitated by computational models that express the processing at each stage of hearing by signal processing. We measure the fundamental functions of hearing such as encoding frequency components, period of waveform, and dynamic range of sounds by psychophysical experimental methods, and reflect the results in a computational model called dynamic compressive gammachirp filterbank model which has been developed in collaborative research projects. By following the prediction of perceptual phenomena by the computational model and its experimental validation, we contribute to the understanding of the human auditory system.

Hearing impairment simulation and its application

Japan is experiencing a “super-aging” society nowadays. It is predicted that age-related hearing loss will increase with the increase in the population of elderly people. Although it is necessary to avoid disconnection of communication due to hearing loss, it is so hard to imagine how hearing loss changes the perception of sound. The computational model of the auditory system can output not only an expression of auditory function, but also sounds deteriorated by modules causing hearing loss. It allows users “to listen as a person with hearing loss”. We are planning to apply this hearing impairment simulation to an educational course for speech therapists, and disperse broadly to the general public to learn about hearing. This simulation will also allow us to evaluate sounds with specific hearing loss, and obtain cues to synthesize clear sounds for all of us.

Changes in perception and cognition by long-term training of music

The questions of how we perceive music as an object with recognizable temporal structure, receive various emotions from it, and enjoy it are still research themes stimulating many researchers. We will study how music changes human audio-visual information processing with the cooperation of professional musicians intensely trained since childhood. We are also investigating the diversity of musicians who have often been treated as a single group

Although humans and animals have great information processing functions, many of such features are yet to be elucidated. Thus, we are studying ways to elucidate the information processing process of those functions and consider a breakthrough based on what we learned from biological information processing when we face an obstacle or difficult problem in artificial information processing. A wide range of academic disciplines must be taken into consideration to examine the information processing mechanism of humans and animals. Therefore, we thoroughly examine physiological and psychological knowledge and proceed with our studies from a multidisciplinary view while taking an information scientific approach. The ultimate goal is to create an artifact with functions superior to that of humans and animals.

Intelligent information processing

We aim to realize the intelligent information processing that humans can perform well and suggest soft computing techniques of neural circuit models, flexible reinforcement learning and self-organizing maps etc.

Sensory information processing model

We explore the mechanism of perception information processing that humans can perform well by applying the modeled visual processing that humans can do well to image processing and modeling the visual process with consideration to the mechanism of processing illusions.

Our research interest lies in the field of systems neuroscience, particularly in the functional relationship between visual perception and neuron activity in the cerebral cortex. The goal of our research is understanding neuronal processes that mediate color perception and object recognition. We have been conducting behavioral and physiological experiments with trained monkeys to perform cognitive tasks. Human psychophysics is also carried out to support correlative evidence between animal behavior and human perception.

Understanding neural basis for visual sensation and cognition

Color is a premier model system for understanding how visual information is processed by neural circuits. Both the physical stimulus for color and the perceptual output experienced as color are quite well characterized, but the neural mechanisms that underlie the transformation from stimulus to perception are incompletely understood. We are focusing on the inferior temporal cortex (ITC), where many neurons response to visual stimuli as highly selective and sophisticated manner. We found a patch organization of color selective cells in the ITC where clusters of neurons showed strong and fine color responses. To understand higher visual function taking place in the ITC such as effect of task demands, memory and utility, color response in the patch could become a useful target area for single unit recording and electrical microstimulation. Human psychophysics is carried out to find out new phenomena, visual illusion and critical feature of visual stimuli. Psychophysical measurement is important to supports correlative evidence between animal behavior and human perception.

Establishing innovative method for neuroscience

EIIRIS have a strong advantage for the development of sensing device such as high density electrode with smart electrical circuits and optical devices. The electrode using Vapor-Liquid-Solid techniques (also known as Toyohashi-probe) is now in trial for effective physiological experiments. An optical imaging system using bundle fibers enable us to monitor functional architecture of deep brain. We debelop these techniques using animalas such as mice, rats, and monkeys.

Behavioral study for dichromatic macaque

Our research groug have found dichromatic macacues a deacade ago, and examined their color vision by genetics (Onishi, et al. 1999), electroretinography (Hanazawa, et al. 2000), and behavioral color discrimination perfromance (Koida, et al. 2013). Futher reseach such as physiological recording in the brain would be expected.

We are conducting researches on underlying the mechanisms of embodied human-human communication and social interaction from multidisciplinary perspectives, such as social robotics, cognitive science and interaction design.

Social robotics and human-robot interaction studies

We use these ecological perspectives to recruit those/things around us to help and build robots with relational capabilities (social robots who use their imperfections as strength). From our research on proto communication, we have created robots such as- Sociable Trash Box: This robot can’t pick up trash on its own, but by cleverly drawing out help from children, the robot picks up trash.

• i-Bones: A robot that nervously draws out the kindness and help from people to give out tissues.
• Talking-Ally: Drawing out help from the listener, Talking-Ally starts conversations and needs the help of the listener to keep the conversation going.
• Muu: Making conversation without words, Muu is a robot who makes conversation with multiple people.

Cognitive science in our everyday communications

Our true research focuses on trying to draw out help from those around us by ways of subtraction design and conversation through minimal spontaneous sounds. We hope that these points of research will lead to new communication methods. Furthermore, our lab looks to find a constructive approach to the aspect of human-human proto-communication. Here are several of our research focuses: – Pursuing the involvement of participants in order to communicate and use minimal cues to bring about communication.

• Developing robots based on isomorphism in relation to the environment and human-robot proto-communication.
• Research on body-body communication through the use of Pelat; the unsteady wandering robot who walks alongside you by holding your hand.

Interaction Design and Social Implementation

By adding more capabilities and raising the functionality of systems, we become more passive and even arrogant at times towards the way we use robots or systems. Our lab in essence, is focused on creating and socially implementing an interactive design which excels in drawing out human kindness, schemes/strategy, and a new way to learn.

• Our research focuses on creating a design which focuses on a relational developmental learning place as well as developing a type of care where if one takes care of something, said person will receive care in return.
• Another focus point is to construct the symbiosis theory through NAMIDA, a level 3 robotic (conditional automation) autonomous driving system. These little agents create a cooperative environment with the driver in which, both NAMIDA and the driver draw out each other’s weakness, while building on those weaknesses to create a stronger bond.

We are developing novel technologies based on artificial intelligence (AI) such as machine learning and neural network, for smartly generating, editing, and analyzing visual contents. Also, mathematical models or applications are developed to create contents or values in a real world, such as digitally designed craftworks or lighting environments, from graphical or image representation.

Humanoid animations based on motion data learning

Classification, recognition, and conversion of human motion data has a big potential for developing digital human technologies in various fields: gestural or behavioral recognitions, simulations for physical trainings, and humanoid animations for video games and VR systems. This project introduces advanced numerical optimizations or machine learning techniques such as deep neural networks to motion data generation or recognition.

Style-based graphics and image processing

Machine learning techniques are rapidly progressed in the research fields of graphics and image synthesis. We are interested in applying state-of-the-art deep learning algorithms to these fields, especially in style manipulation/conversion of graphical/image representations such as motion capture data and illustrative or cartoon-like images.

Image-based craftwork and optical control

This project develops the methodologies or novel applications based on artificial or illustrative images. Smart image processing such as conversion, sampling, style recognition, is developed to support the designs and creations of physical object, such as machine embroidery and smart controls of decorative illuminations, using state-of-the-art methods of machine learnings.

We are developing novel technologies based on artificial intelligence (AI) such as machine learning and neural network, for smartly generating, editing, and analyzing visual contents. Also, mathematical models or applications are developed to create contents or values in a real world, such as digitally designed craftworks or lighting environments, from graphical or image representation.

Humanoid animations based on motion data learning

We aim to develop intelligent systems, such as service robots and self-driving vehicles, which can operate autonomously and intelligently in complex real environments. A key to realize such systems is advanced information processing or AI, including scene recognition, context-aware planning, and human-machine interaction.

Attendant robot / Service robot

Autonomous service robots are expected to support our daily life in various scenarios, such as attending, guiding, errand, and search. We have been developing novel methods for such service robots, including robust scene recognition, human detection/tracking/identification and pose estimation, and task and action planning.

Outdoor navigation / Autonomous driving

Outdoor navigation is one of the necessary functions of service robots since outdoor environments are important parts of human activity. We have been developing novel methods for outdoor navigation such as multi-sensory road boundary tracking, view-based robust localization, and large-scale 3D mapping and localization. Many of these methods can be applied to autonomous driving.

Human-robot interaction

Human-robot collaboration is an effective way of achieving non-trivial tasks in complex real environments. We have been developing novel approaches in various collaborating scenarios such as human-robot collaborative assembly, collaborative remote object search, and robot-mediated task knowledge transfer.

Our laboratory aims to a system that support our daily life using ubiquitous technologies. We study several techniques in wide range from devices to AI technologies, such as embedded systems, wearable computers, sensor networks, computer networks, distributed systems, system software, pattern recognition, visualization and so on.

Body (Wearable) Scale System

In recent days, wearable devices, such as a smart watch and a head mounted display, are getting common. Connecting these devices as a “body-area network”, we study techniques to automatically understand what a user is doing now, so-called “activity recognition”. Also, using the recognition results, we also develop systems to support human activities, such as training of certain action form, error detection of tasks. Figure 1 shows the example of cardio pulmonary resuscitation (CPR) support system using wristworn sensors that gives instruction of correct CPR on the site of emergency rescue.

House and Office Scales Systems

Now, appliances in a house and office, such as a TV, air conditioner, and a cleaning robot, have functions of information processing. Activity and situation recognition techniques described above can be extended to these environment, and we develop a system that support our daily life at our house and working environment, such as a system supporting efficient and safe medical care in hospitals. Figure 2 shows a research project using Terapio, a robot supporting medical rounding in a hospital, collaborating with Human-Robot Symbiosis Research Center at TUT. Also, we are developing an application execution and development environment of home network that enables ordinal people to build a program easily, by personifying appliances and integrating between-human and betweendevice communication. Moreover, we also develop novel sensor devices which works efficiently with energy-harvesting and wireless power transmission techniques.

Urban Scale Systems

Mobile phones and wearable computers accomplish the support of our activity even outside environment. In other words, these devices can contribute to design our city in the aspect of information, and our laboratory aims to a “smart city” where citizens can spend their happy, enjoyable, and safe life. One example of the systems we are developing is the one supporting emergency rescue enabling efficient information exchange among concerning organizations, such as fire department, ambulance, and hospitals. Also, Figure 3 shows another example, a navigation system of a city and theme park that reflects users’ impression by detecting their emotion with wearable devices automatically.

The aim of computer vision is to realize the functions of human vision on a computer and its applications are widely used for various fields: 3-D measurement, surveillance system, robot vision, medical image processing, and so on. In this laboratory, we focus on 3-D reconstruction from images and matching between images. We also study image processing for colorblind persons and a safety system for vulnerable road users like children and elderly persons using image processing.

Accurate image matching

Image matching is the first step for many computer vision applications like 3-D reconstruction and object recognition. The accuracy affects to that of reconstructed 3-D shape. We are studying robust and accurate image matching methods for various scenes.

3-D reconstruction from images under

special environments Endoscope image sequence is one of difficult case for accurate 3-D reconstruction because of the special motion of the camera. Drone images of the sands or beach is also difficult to obtain accurate 3-D shape because the target is almost planar and its texture is almost the same. We are conducting the research of accurate 3-D reconstruction under such special environments.

Image enhancement for colorblind

person Colorblind person feels inconvenience in daily life because the color design of almost things is not adequate for them, e.g., traffic signs/signals, road/ floor maps, even web pages. We have proposed image enhancing method by additive image noise for them. By our method, not only colorblind person can distinguish their indistinguishable colors but also normal person can recognize the original colors. We are conducting the improvement of our method.

We are studying geometric problems of computer vision including 3-D reconstruction, ellipse fitting, ellipse detection, and mixed reality system.

Mixed reality system using circular marker

We are studying ellipse detection from images and use this technique for detecting a circular marker of a mixed reality system. The advantage of using a circular marker is that we can estimate a camera position and pose from this marker if it is hidden by other objects.

Developing diminished reality system

We are developing a diminished reality system using circular markers and a 3-D shape of a background scene obtained from a RGB-D sensor like the Kinect. We are now focusing an alignment between the overlaid background and a captured image from a user camera.

Ellipse fitting and ellipse detection

We are studying a high-accuracy ellipse fitting technique and ellipse detection from images for applying mixed reality system and camera calibration.

 

Currently, we are developing several communication robots that can involve in conversation with people using verbal and non-verbal means of communication. Through our research, we intend to construct a suitable robot architecture for achieving human-like multi-party conversation among more than two robots. In this study, we are constructing and evaluating the social interactions of robots that can participate in human communications. Therefore, our research findings will help to understand the daily communication mechanisms of humans.

Design of a Speech Eliciting Robot that Intervenes in Brainstorming Sessions to Ensure Collaborative Group Work

For brainstorming to work effectively, a cooperative atmosphere in which all participants can exchange their ideas in a collaborative manner is essential. On such occasions, situations in which the topic of discussion moves on without some participants being able to share their comments or ideas must be obviated. In this research, we developed a speech eliciting robot (Neut) that ensures a cooperative brainstorming environment. Neut creates an atmosphere that makes it easier for participants who are often overlooked to express their ideas, by promoting cooperation from the other participants. Neut is a small robot that moves freely on a table and approaches one or the other participant who has not yet had his/her speaking turn. After stopping in front of such a participant, it brings out a wireless microphone and prompts the participant to speak, while looking around restlessly to suggest to others that they give the participant a chance to speak.

Bot-type Agents to Promote Conversations Between a Child and a Parent Who Live Far Away

Daily communication between a child and the parent living alone is important for maintaining parent-child relationships and verification of health. However, a parent and a child living away from one another will only contact each other when there is a special event or reason, and communication will gradually fade. The purpose of this study is to construct a system to promote parent-child remote communication. Therefore, based on a parent’s cooking and chat activity of multiple agents, a communication support system that gives a parent and a child the opportunity to speak has been developed.