Reviewed (English)

Object-Centric Representation Learning with Attention Mechanism [abst]
Hidemoto Nakada, Hideki Asoh , The 18th International Conference on Ubiquitous Information Management and Communication (IMCOM '24) , 2024

For object-centric representation learning, several slot-based methods, that separate objects using masks and learn the objects separately, are proposed. While these methods are proved to be useful on various downstream tasks, it is known that they require a significant amount of computation for training. We propose the introduction of attention mechanisms into slot-based method to simplify and speed up the computation. We pick ViMON as the base structure and propose two methods, named AttnViMON and SFA. We evaluate them in terms of reconstruction error and computation time, and a downstream task. The proposed methods demonstrate that they achieve significant speed-up while showing even better performance.

Cloud-based Testbed for Large-scale data Collection System with Network-Edge [abst]
Yunzhi Dong , Hidemoto Nakada, Yusuke Tanimura , The 18th International Conference on Ubiquitous Information Management and Communication (IMCOM '24) , 2024

With the recent development of IoT technology, it is becoming necessary to design systems that utilize not only the cloud but also the edge. We are studying systems that reduce the load on the cloud by performing various processes at the edge. Although experimentation in a large-scale environment is essential for such research, it is impractical to set up such an environment using actual computers. In this study, we attempted to construct a large-scale experimental environment on top of EKS, a container framework on Amazon Web Service and confirmed that it is possible to easily construct a large-scale experimental environment by using containers. Furthermore, we conducted several experiments on the constructed large-scale experimental environment and confirmed the following: 1) The computational load for the MQTT broker is sufficiently small even with high message frequency, 2) With a relay server, we can reduce the broker burden preserving the message delivery latency.

Improve symbolic music pre-training model using MusicTransformer structure [abst]
Yingfeng Fu, Yusuke Tanimura, Hidemoto Nakada , The 17th International Conference on Ubiquitous Information Management and Communication , 2023

Pre-training driven by vast data has shown great power in natural language understanding. The idea has also been applied to symbolic music. However, many existing works using pre-training for symbolic music are not general enough to tackle all the tasks in musical information retrieval, and there is still space to improve the model structure. To make up for the insufficiency and compare it with the existing works, we employed a BERT-like masked language pre-training approach to train a stacked MusicTransformer on MAESTRO dataset. Then we fine-tuned our pre-trained model on several symbolic music understanding tasks. In the work, our contribution is 1)we improved MusicBERT by modifying the model structure. 2)besides the existing evaluation downstream tasks, we complemented several downstream tasks, including melody extraction, emotion classification, and composer classification. We pre-trained the modified model and existing works under the same condition. We make a comparison of our pre-trained model with the previous works. The result shows that the modified model is more powerful than the previous models with the same pre-training setting.

End-To-End Training Of Object Segmentation Task And Video Question-Answering Task [abst]
Hidemoto Nakada, Hideki Asoh , The 17th International Conference on Ubiquitous Information Management and Communication , 2023

For complicated VQA tasks that incorporates multiple objects, to train the VQA model using segmented objects data as inputs is proved to be effective for various downstream tasks. In this work, we tried to train the VQA task model and object segmentation model in end-to-end fashion instead of training independently. We employed CLEVRER as a target VQA task. We first trained MONet, an object segmentation network, with the dataset, and trained Aloe, a VQA model, using the output of the trained MONet. Finally we connect MONet and Aloe to fine-tune them in end-to-end setting and confirmed that the performance of VQA task has been greatly improved.

CloudQ: A Secure AI / HPC Cloud Bursting System [Paper] [Slides] [abst]
Shin'ichiro Takizawa, Masaaki Shimizu, Hidemoto Nakada, Hiroya Matsuba, Ryousei Takano , 9th International Workshop on HPC User Support Tools , pp. 48 - 50 , 2022

As a method to optimize the investment for computational resources, cloud bursting is collecting a lot of attention, where the organizations utilize the cloud computing environment in on-demand fashion, while preserving the minimum amount of on-premise resources for sensitive data processing. For the practical cloud bursting, we need to achieve 1) secure job / data sharing, 2) uniform job execution environment for on-premise and cloud, and 3) on-demand automatic deployment of the execution environment on the cloud. To enable these items, we propose a meta-scheduling system called CloudQ. CloudQ 1) uses cloud object storage for data sharing, 2) utilizes container images to provide uniform job execution environment, and 3) automatically deploys an execution environment on the cloud.

Automated Quantization and Retraining for Neural Network Models without Labeled Data [abst]
Kundjanasith Thonglek, Keichi Takahashi, Kohei Ichikawa, Chawanat Nakasan, Hidemoto Nakada, Ryousei Takano, Pattara Leelaprute, Hajimu Iida , IEEE Access, Vol.10 , pp. 73818-73834 , 2022

Deploying neural network models to edge devices is becoming increasingly popular because such deployment decreases the response time and ensures better data privacy of services. However, running large models on edge devices poses challenges because of limited computing resources and storage space. Researchers have therefore proposed various model compression methods to reduce the model size. To balance the trade-off between model size and accuracy, conventional model compression methods require manual effort to find the optimal configuration that reduces the model size without significant degradation of accuracy. In this article, we propose a method to automatically find the optimal configurations for quantization. The proposed method suggests multiple compression configurations that produce models with different size and accuracy, from which users can select the configurations that suit their use cases. Additionally, we propose a retraining method that does not require any labeled datasets for retraining. We evaluated the proposed method using various neural network models for classification, regression and semantic similarity tasks, and demonstrated that the proposed method reduced the size of models by at least 30% while maintaining less than 1% loss of accuracy.We compared the proposed method with state-of-the-art automated compression methods, and showed that it can provide better compression configurations than existing methods.

Performance of Domain Adaptation Schemes in Video Action Recognition using Synthetic Data [abst]
Hana Isoi, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , The 2022 4th International Conference on Image, Video and Signal Processing (IVSP 2022) , pp. 70-79 , 2022

The videos obtained from surveillance cameras are expected to be processed in neural networks and utilized for indoor monitoring and surveillance services. However, collecting training data is generally an issue in machine learning, obtaining sufficient learning data is very difficult from the viewpoint of privacy protection, especially for the abovementioned services. To address this issue, synthetic data instead of real data might be employed for training. However, the domain shift of video has not been sufficiently investigated, and therefore high-accuracy, unsupervised learning of real video using synthetic video has not been achieved. In this study, we create a synthetic video dataset for action classification and show that action classification can be performed with high accuracy without labeling real video data by learning with this synthetic dataset. First, we create an OchaHouse Dataset for action classification for indoor monitoring and surveillance. This dataset consists of a real video dataset OchaHouse-Real that records the behavior of one person in a room and a synthetic video dataset OchaHouse-Syn that resembles it. Second, video action recognition of real videos is performed using synthetic data and various unsupervised learning methods with domain adaptation schemes. The results show that learning with synthetic data enables highly accurate action classification of real data without a real data label. The dataset will be published.

A Method to Generate Posed Person Image with few Context Images [abst]
Hidemoto Nakada, Hideki Asoh , IMCOM 2022, The 16th International Conference on Ubiquitous Information Management and Communication , 2022

The method we propose creates a pose invariant person representation using the attention mechanism and generates posed images by applying pose query on the representation. We evaluated the method using 3D rendered synthetic data and confirmed that the created person representation is pose-invariant, and we can render good quality images with the representation.

Few Shot Model based on Weight Imprinting with Multiple Projection Head [abst]
Paulino Cristovao, Hidemoto Nakada, Yusuke Tanimura, Hideki Asoh , IMCOM 2022, The 16th International Conference on Ubiquitous Information Management and Communication , 2022

Few shot learning models based on imprinted weights have achieved excellent results on the low-data regime. In these methods the network directly sets the weight of the final layers for novel classes from the embeddings. As a result, the embeddings learned lead to the highest classification accuracy on base classes; however, the classification accuracy might be degraded on the novel classes. In this paper, we provide an efficient training approach for imprinted weight models. We find that a simple design choice of imprinted weights can yield substantial improvements over the baseline model. Our experiments show that (1) introducing a nonlinear projection heads in-between feature extractor, and classifier substantially improves generalization, (2) imprinting from the last projection head does not provide better generalization for novel classes. Instead, we propose imprinting from optimal projection head, and (3) this design choice benefits from a large latent dimension. We validate our findings by achieving 5.6 and 4.1\% improvement on MNIST dataset trained with Omniglot dataset.

Variational Information Bottleneck on Few Shot Model based on Weight Imprinting for Image Classification [abst]
Paulino Cristovao, Hidemoto Nakada, Yusuke Tanimura, Hideki Asoh , 2021 ASIAN CONFERENCE ON INNOVATION IN TECHNOLOGY , 2021

Few shot learning remains an open issue in computer vision. Among several recently proposed approaches, Weight Imprinting (WI) achieves superior performance on many challenging benchmarks. The performance of the imprinted weights heavily depends on the quality of the representations generated by the encoder. However, it is not known what characteristics are required for weight imprinting. The representations learned during the pre-training phase are optimized for classification accuracy for the pre-training base classes and not necessarily suitable for the downstream, imprinted tasks. In this paper, we investigate the effect of \VIB on the few shot learning with weight imprinting. \VIB strongly regularizes the representation by minimizing the mutual information between input data and representation while keeping the classification accuracy for pre-training task. We demonstrate that the encoder regularized by VIB achieves significantly better performance on few-shot learning tasks with imprinting. Furthermore, we comprehensively investigate the effect of combining VIB with other regularization methods including data augmentation and auxiliary data. We confirmed that with a proper auxiliary dataset, we can achieve even better accuracy on the downstream task.

One-shot style transfer using Wasserstein Autoencoder [Paper] [abst]
Hidemoto Nakada, Hideki Asoh , 2021 ASIAN CONFERENCE ON INNOVATION IN TECHNOLOGY , 2021

We propose an image style transfer method based on disentangled representation obtained with Wasserstein Autoencoder. Style transfer is an area of image generation technique that generates an image that shows content taken from one image with a style taken from another image. While there are extensive researches in this area, most of them require some ’training’ time to generate images with a specific style. The proposed method does not require training time since we train a versatile network that can be used for any style and content image. The network encodes images into disentangled latent variables that represent content and style. We can transfer image style by simply replacing style latent variables. We tested the proposed method with images from CelebA and confirmed that it can generate style transferred images.

Action Recognition using Pose Data in a Distributed Environment over the Edge and Cloud [abst]
Chikako Takasaki, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , IEICE Transactions on information and systems, Vol.E104-D, No.5 , pp. 539-550 , 2021

With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving imagesthat are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacyconcerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.

Generating In-Between Images Through Learned Latent Space Representation Using Variational Autoencoders [abst]
Paulino Cristovao, Hidemoto Nakada, Yusuke Tanimura, Hideki Asoh , IEEE Access, vol 8, , pp. 149456-149467 , 2020

Image interpolation is often implemented using one of two methods: optical flow or convolutional neural networks. These methods are typically pixel-based; they do not work well on objects between images far apart. Because they either rely on a simple frame average or pixel motion, they do not have the required knowledge of the semantic structure of the data. In this paper, we propose a method for image interpolation based on latent representations. We use a simple network structure based on a variational autoencoder and an adjustable hyperparameter that imposes the latent space distribution to generate accurate interpolation. To visualize the effects of the proposed approach, we evaluate a synthetic dataset. We demonstrate that our method outperforms both pixel-based methods and a conventional variational autoencoder, with particular improvements in nonsuccessive images.

Retraining Quantized Neural Network Models with Unlabeled Data [abst]
Kundjanasith Thonglek, Keichi Takahashi, Kohei Ichikawa, Chawanat Nakasan, Ryousei Takano, Hidemoto Nakada, Hajimu Iida , International Joint Conference on Neural Networks (IJCNN) , 2020

Running neural network models on edge devices is attracting much attention by neural network researchers since edge computing technology is becoming more powerful than ever. However, deploying large neural network models on edge devices is challenging due to the limitation in available computing resources and storage space. Therefore, model compression techniques have been recently studied to reduce the model size and fit models on resource-limited edge devices. Compressing neural network models reduces the size of a model, but also degrades the accuracy of the model since it reduces the precision of weights in the model. Consequently, a retraining method is required to recover the accuracy of compressed models. Most existing retraining methods require the original labeled training datasets to retrain the models, but labeling is a time-consuming process. In particular, we cannot always access the original labeled datasets because of privacy policies and license limitations. In this paper, we propose a method to retrain a compressed neural network model with an unlabeled dataset that is different from the original labeled dataset. We compress the neural network model using quantization to decrease the size of the model. Subsequently, the compressed model is retrained by our proposed retraining method without using a labeled dataset to recover the accuracy of the model. We compared the proposed retraining method against the conventional retraining. The proposed method reduced the size of VGG-16 and ResNet-50 by 81.10% and 52.45%, respectively without significant accuracy loss. In addition, our proposed retraining method is clearly faster than the conventional retraining method.

One-Shot Learning Using Triplet Network with kNN Classifier [abst]
Mu Zhou, Yusuke Tanimura, Hidemoto Nakada , JSAI 2019: Advances in Artificial Intelligence, vol 1128. Springer, , pp. 227-235 , 2020

This is an extension from a selected paper from JSAI2019. Humans have the ability to learn new things correctly without requiring large amount of data, while it is a challenging task in AI, which is called few-shot Learning or one-shot learning. Our key insight is using data augmentation technique to enlarge our dataset, then feeding them into a Triplet Network which is to collect same categories and separate the different. We have compared different augmentation methods, and we confirm that CVAE(Conditional VAE) can make sense as data augmentation method to slove one-shot classification problems.

A Study of Action Recognition using Pose Data toward Distributed Processing over Edge and Cloud [abst]
Chikako Takasaki, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , Proc. of CloudCom 2019 , 2019

With the development of cameras and sensors, and the spread of cloud computing, life logs can be acquired and stored in general households for various services using the logs. However, it is difficult to analyze moving images acquired by a home sensor in real time using machine learning because the data size and the computational complexity are large. New computing paradigm called edge computing or fog computing, which enables distributed computing over edge and cloud, has the possibility to address this issue. The feature vectors are extracted from moving images by preprocessing on the sensor side and the only small feature vectors are sent to the cloud and used for learning. But, it is not clear how accurately we can recognize actions using only the feature vectors in the learning and inferring. We investigate the accuracies of action recognition with various machine learning methods using feature vector information obtained from moving images. We use the pose estimation library OpenPose for detection of the feature vectors and recognize actions using logistic regression, random forest, support vector machine, and neural network (NN) models, general NN and LSTM, as machine learning methods. The experimental results show that it is possible to recognize an action with 80% accuracy or higher when using random forest and neural network models. We also discuss a method to further improve the accuracy based on the experimental results.

Hierarchical Reinforcement Learning with Unlimited Recursive Subroutine Calls [abst]
Yuuji Ichisugi, Naoto Takahashi, Hidemoto Nakada, Takashi Sano , 28th International Conference on Artificial Neural Networks (ICANN 2019) , 2019

We propose a new hierarchical reinforcement learning architecture called the RGoal architecture.RGoal solves the Markov Decision Process (MDP) in an augmented state-action space.

Construction Scheme of a Scalable Distributed Stream Processing Infrastructure Using Ray and Apache Kafka [abst]
Kasumi Kato, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , Proc. of 34th International Conference on Computers and Their Applications(CATA 2019), EPiC Series in Computing, vol. 58 , pp. 368–377 , 2019

The spread of various sensors and the development of cloud computing technologies en- able the accumulation and use of large numbers of live logs in ordinary homes. To operate a service that utilizes sensor data, it is difficult to install servers and storage in ordinary homes and to analyze the collected data from sensors. Those data are typically transmitted from sensors to a cloud and analyzed in the cloud. However, services that involve moving image analysis must transfer large amounts of data continuously and require high computing power for analysis. Hence, it is highly difficult to process them in real time in the cloud using a conventional stream data processing framework. In this research, we propose a construction scheme for a highly efficient distributed stream processing infrastructure that enables scalable processing of moving image recognition tasks according to the amount of data that are transmitted from sensors. We implement a prototype system of the proposed distributed stream processing infrastructure using Ray and Apache Kafka, which is a distributed messaging system, and we evaluate its performance. The experimental results demonstrate that the proposed distributed stream processing infrastructure is highly scalable.

Performance Evaluation of Pipeline-Based Processing for the Caffe Deep Learning Framework [abst]
Ayae Ichinose, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , IEICE Trans. on Information & System, Vol.E101-D No.4 , pp. 1042-1052 , 2018

Many life-log analysis applications, which transfer data from cameras and sensors to a Cloud and analyze them in the Cloud, have been developed as the use of various sensors and Cloud computing tech- nologies has spread. However, difficulties arise because of the limited net- work bandwidth between such sensors and the Cloud. In addition, sending raw sensor data to a Cloud may introduce privacy issues. Therefore, we propose a pipelined method for distributed deep learning processing be- tween sensors and the Cloud to reduce the amount of data sent to the Cloud and protect the privacy of users. In this study, we measured the process- ing times and evaluated the performance of our method using two differ- ent datasets. In addition, we performed experiments using three types of machines with different performance characteristics on the client side and compared the processing times. The experimental results show that the accuracy of deep learning with coarse-grained data is comparable to that achieved with the default parameter settings, and the proposed distributed processing method has performance advantages in cases of insufficient net- work bandwidth between realistic sensors and a Cloud environment. In addition, it is confirmed that the process that most affects the overall pro- cessing time varies depending on the machine performance on the client side, and the most efficient distribution method similarly differs.

Understanding and Improving Disk-based Intermediate Data Caching in Spark [abst]
Kaihui Zhang, Yusuke Tanimura, Hidemoto Nakada, Hirotaka Ogawa , Scalable Cloud Data Management Workshop 2017 in IEEE BigData , pp. 2426-2435 , 2017

Apache Spark is a parallel data processing framework that executes fast for iterative calculations and interactive processing, by caching intermediate data in memory with a lineage-based data recovery from faults.  The Spark system can also manage data sets larger than memory capacity by placing some cache or all of them on disks on processing nodes.  The disadvantage, however, is potential performance degradation due to disk I/O and/or serialization.  This study aims to clarify efficient/inefficient use of disks in intermediate data caching in Spark and also to improve the usability of disks for end users.  In order to achieve the purpose, influence of disk use in data caching was firstly investigated in various aspects, such as caching options, data abstractions and storage devices.  The results indicate that serialization cost is dominant rather than disk I/O in most cases.  Secondly, a method of combined use of memory and disk was further evaluated under a high memory pressure.  Then the method was improved to avoid an excessive re-caching problem, which achieved at most 20-30\% reduction of total execution time under a high memory pressure and did not degrade the performance under a low memory pressure, in our experiment with 4 machine learning benchmarks.  Finally, this paper summarizes important factors and potential improvements for efficiently using disks in data caching in Spark.

A Study of a Video Analysis Framework Using Kafka and Spark Streaming [abst]
Ayae Ichinose, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , Second workshop on Real-time and stream processing in IEEE BigData 2017 , pp. 2314-2319 , 2017

The use of various sensors and cloud computingtechnologies has spread, many life-log analysis applications forsafety services for the elderly and children have been developed.However, it is difficult to perform real-time large data processingin clouds due to the computational complexity of the analysisbecause efficient deployment schemes of streaming computingcomponents over cloud resources have not been well-investigated.In this study, we propose a video analysis framework that collectsvideos from multiple cameras and analyzes them using ApacheKafka and Apache Spark Streaming. We first investigate the datatransfer performance of Apache Kafka and examine efficientcluster configuration and parameter settings. We then applythis configuration to the proposed framework and measure thedata analysis throughput. The experimental results show that theoverall throughput varies depending on the number of brokernodes that store data, the number of topic partitions of data,and the number of nodes that conduct analysis processing. Inaddition, it is confirmed that the number of cores is neededto consider for the efficient cluster configuration, and that thenetwork bandwidth between the nodes becomes a bottleneck asthe amount of data and the number of components increase.

How Much Should We Invest for Network Facility: Quantitative Analysis on Network 'Fatness' and Machine Learning Performance [Paper] [Slides] [abst]
Duo Zhang, Mingxi LI, Yusuke Tanimura, Hidemoto Nakada , Workshop on ML Systems in NIPS 2017 , 2017

Multi-node execution is becoming more and more popular for machine learning because of it's huge amount of computation. The question we are trying to answer here is that, how should we design computer systems for deep learning, especially in terms of investment for the network. Traditional cluster based 'super-computers' require huge amount of investment on network switches since the network 'fatness' is quite important for the typical applications of super-computers. Do the machine learning workloads share the characteristics with such kinds of applications? To answer this questions, we quantitatively analyze the impact of network fatness on several type of machine learning application types with several network configurations. The results we obtained strongly implies that the network fatness is not important for machine learning applications, and thus we could safely reduce investment on network facilities.

A Quantitative Analysis on Required Network Bandwidth for Large-Scale Parallel Machine Learning [abst]
Mingxi Li, Yusuke Tanimura, Hidemoto Nakada , MOD 2017 - (The Third International Conference on Machine Learning, Optimization and Big Data) , LNCS vol.10710 , pp. 389-400 , 2017

Parallelization is essential for machine learning systems that deals with large-scale data-set. Data parallel machine leaning systems, that is composed of multiple machine learning modules, exchange the parameter to synchronize the models in the modules, via network. We investigate the network bandwidth requirements for various parameter exchange method, using cluster simulator SimGrid. We have confirmed that 1) direct exchange method substantially more efficient than parameter server based method, and 2) with proper exchange methods, the bisection-bandwidth of the network does not affect the efficiency, allowing smaller investment on network facility.

Context-Dependent Robust Text Recognition using Large-scale Restricted Bayesian Network [abst]
Hidemoto Nakada, Yuuji Ichisugi , BICA2017(2017 Annual International Conference on Biologically Inspired Cognitive Architectures), Procedia Computer Science, Volume 123, 2018, Elsevior , pp. 314-320 , 2017

We have been proposing a computational model of the cerebral cortex called BESOM, which models the cerebral cortex as restricted Bayesian networks based on recent findings in the neuroscience area. Since BESOM is based on Bayesian network, it inherently allows bi-directional information flow, meaning that it can naturally merge information extracted from concrete data with highly-abstract prior knowledge. As an example of such kind of tasks, we report robust text recognition task with context information. We show that word spelling knowledge and word n-gram could be represented as a part of the network and they contribute the text recognition accuracy with noisy text images.We also show that the computational cost is approximately linear with the number of characters and words.

Pipeline-based Processing of the Deep Learning Framework Caffe [abst]
Ayae Ichinose, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , ACM IMCOM 2017 , 2017

Many life-log analysis applications, which transfer data from cameras and sensors to a Cloud and analyze them in the Cloud, have been developed with the spread of various sensors and Cloud computing technologies. However, difficulties arise because of the limitation of the network bandwidth between the sensors and the Cloud.In addition, sending raw sensor data to a Cloud may introduce privacy issues. Therefore, we propose distributed deep learning processing between sensors and the Cloud in a pipeline manner to reduce the amount of data sent to the Cloud and protect the privacy of the users. In this paper, we have developed a pipeline-based distributed processing method for the Caffe deep learning framework and investigated the processing times of the classification by varying a division point and the parameters of the network models using data sets, CIFAR-10 and ImageNet. The experiments show that the accuracy of deep learning with coarse-grain data is comparable to that with the default parameter settings, and the proposed distributed processing method has performance advantages in cases of insufficient network bandwidth with actual sensors and a Cloud environment.

A Quantitative Analysis of Fault Tolerance Mechanisms for Parallel Machine Learning Systems with Parameter Servers [abst]
Mingxi Li, Yuusuke Tanimura, Hidemoto Nakada , ACM IMCOM 2017 , 2017

Parallel computation is essential for machine learning systems that handle large amount of data for training. One popular form of parallel machine learning is the one called 'Data parallel' which use large number of parameter servers that manage computational workers. Fault tolerance is always a crucial issue on large scale computation systems in general, and parameter server based machine learning systems are no exception. However, there are many discussions on the fault tolerance of large scale computation systems in general, there are no discussions on parallel machine learning systems, in spite of their unique characteristics. In this paper, we discuss the fault tolerance of parallel machine learning systems which use parameter servers that provides extra redundancy to the system and could double as the checkpoint server. We also quantitatively evaluate several fault tolerance method using parallel environment simulator SimGrid, and demonstrate the effectiveness of the proposed method.

A Study of Load Balancing between Sensors and a Cloud for a Real Time Video Streaming Analysis Application Framework [abst]
Yuko Kurosaki, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , ACM IMCOM 2016 , 2016

For the home and office, many life-log analysis applications for transferring data from cameras and sensors to the cloud and analyzing the data have been developed. However, because of limitation of the resources of the cloud and the network bandwidth between the sensor and the cloud, it is difficult to execute large load processing, such as video streaming analysis, in real time on the cloud. Moreover, taking into account the execution environment from the sensor to the cloud, it is necessary to set an appropriate degree of parallelism in the processing from the pre-processing, such as feature extraction, to the analysis on the information. In this paper, we propose a video streaming analysis application framework for load balancing between sensors and the cloud, and investigate the performance of the application in a cluster environment that simulates the sensor and the cloud. From the experiments, we show that the processing performance is improved by increasing the number of processing threads, and we demonstrate the effectiveness of load balancing between the sensors and the cloud.

Surface object recognition with CNN and SVM in Landsat 8 images [abst]
Tomohiro Ishii, Ryosuke Nakamura, Hidemoto Nakada, Yoshihiko Mochizuki, Hiroshi Ishikawa , 2015 14th IAPR International Conference on Machine Vision Applications (MVA) , 2015

There is a series of earth observation satellites called Landsat, which send a very large amount of image data every day such that it is hard to analyze manually. Thus an effective application of machine learning techniques to automatically analyze such data is called for. In surface object recognition, which is one of the important applications of such data, the distribution of a specific object on the surface is surveyed. In this paper, we propose and compare two methods for surface object recognition, one using the convolutional neural network (CNN) and the other support vector machine (SVM). In our experiments, CNN showed higher performance than SVM. In addition, we observed that the number of negative samples have a influence on the performance, and it is necessary to select the number of them for practical use.

A Highly Available Distributed Self-Scheduler for Exascale Computing [abst]
Atsuko Takefusa, Hidemoto Nakada, Tsutomu Ikegami, Yoshio Tanaka , IMCOM (ICUIMC) 2015 , 2015

A hierarchical master-worker model is thought to be a promising programming paradigm for exascale-level high performance computers. However, ``fault resiliency'' is one of the most important issues for exascale computing because the Mean Time Between Failure (MTBF) is expected to be short. We propose a fault resilient middleware suite for exascale computing environments. In this paper, we design a highly available distributed self-scheduler as a resource management system for the proposed middleware suite. The proposed distributed self-scheduler consists of multiple processes in order to achieve scalability, fault resiliency, and persistency. We also develop a prototype system of the middleware, using Apache ZooKeeper and Apache Cassandra. Experiments using the developed prototype system show that the proposed distributed self-scheduler achieves the desired fault resiliency for an application program developed using the middleware, and that the scheduler itself is also fault resilient. We also confirmed that the overheads caused by distributed processing can be reduced, and the scheduler can be scalable.

A Study of Replica Reconstruction Schemes for Multi-rack HDFS Clusters [abst]
Asami Higai, Atsuko Takefusa, Masato Oguchi, Hidemoto Nakada , 7th IEEE/ACM International Conference on Utility and Cloud Computing (UCC 2014) , 2014

Distributed file systems, which enable users to manage large amounts of data over multiple commodity computers, have attracted attention as a potential management and processing system for big data applications. The Hadoop Distributed File System (HDFS) is a widely used open source distributed file system. In the HDFS, multiple replicas are separately stored over the multiple data nodes for enhanced availability. When a data node failure is detected, replica reconstruction is performed. During this process, the access load of the other data nodes, which hold the lost data blocks, may increase, so that the overall performance of data processing over the distributed file system decreases. Therefore, an important issue is effective replica reconstruction in order to prevent such performance degradation. In addition, HDFS composed of multiple racks is needed to replicate the missing blocks on a different rack according to the HDFS replica placement policy, for the purpose of availability. We have to take into account network bandwidth and fault tolerance for such blocks which require data transfer between racks in the cluster. In this paper, we propose replica reconstruction schemes for a multi-rack HDFS cluster and evaluate the effectiveness of our proposed schemes in multi-rack cluster environments by simulation. In the proposed schemes, data transfer in a rack is performed based on a one-directional ring structure and inter-rack data transfer is performed in a round robin manner. We control streams between racks as giving the priority for the blocks which requires inter-rack transfer. The experiments show that the proposed schemes are effective for reduction of the execution time and improvement of the fault tolerance. We also confirm that the performance shows further improvement by controlling the number of streams between racks properly and the execution times of our proposed schemes show a 16 % reduction in time required compared to that of the default scheme.

A Study of Effective Replica Generation Schemes at Node Deletion for HDFS [abst]
Asami Higai, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , Proc. IEEE/ACM CCGrid 2014 , pp. 512-521 , 2014

Distributed file systems have attracted attention as a management and processing system for big data applications. A distributed file system consists of multiple data nodes and provides reliability and availability by holding multiple replicas of data. Due to system failure or maintenance, a data node may be removed from the system and the data blocks the removed data node held are lost. If data blocks are missing, the access load of the other data nodes that hold the lost data blocks increases, and as a result the performance of data processing over the distributed file system decreases. Therefore, replica reconstruction is an important issue to reallocate the missing data blocks in order to prevent such performance degradation. The Hadoop Distributed File System (HDFS) is a widely used distributed file system. In the HDFS replica reconstruction process, source and destination data nodes for replication are selected randomly. We found that this replica reconstruction scheme is inefficient because data transfer is biased. Therefore, we propose two more effective replica reconstruction schemes that aim to balance the workloads of replication processes. Our proposed replication scheduling strategy assumes that nodes are arranged in a ring and data blocks are transferred based on this one-directional ring structure to minimize the difference of the amount of transfer data of each node. Based on this strategy, we propose two replica reconstruction schemes, an optimization scheme anda heuristic scheme.We have implemented the proposed schemes in HDFS and evaluated them on an actual HDFS cluster. From the experiments, we confirm that the replica reconstruction throughput of the proposed schemes show a 45% improvement compared to that of the default scheme. We also verify that the heuristic scheme is effective because it shows performance comparable to the optimization scheme and can be

Iris: An Inter-cloud Resource Integration System for Elastic Cloud Data Centers [abst]
Ryousei Takano, Atsuko Takefusa, Hidemoto Nakada, Seiya Yanagita, Tomohiro Kudoh , Proc. of CLOSER 2014 , pp. 103-111 , 2014

We propose a new cloud computing service model, Hardware as a Service (HaaS), that is based on the idea of implementing “elastic data centers” that provide a data center administrator with resources located at different data centers as demand requires. To demonstrate the feasibility of the proposed model, we have developed what we call an Inter-cloud Resource Integration System (Iris) by using nested virtualization and OpenFlow technologies. Iris dynamically configures and provides a virtual infrastructure over inter-cloud resources, on which an IaaS cloud can run. Using Iris, we have confirmed an IaaS cloud can seamlessly extend and manage resources over multiple data centers. The experimental results on an emulated inter-cloud environment show that the overheads of the HaaS layer are acceptable when the network latency is less than 10 ms. In addition, we reveal the large overhead from nested virtualization and show positive prospect for this problem. We believe these results provide new insight to help establish inter-cloud computing.

Implementation of Data Affinity-based Distributed Parallel Processing on a Distributed Key Value Store [abst]
Naoko Hishinuma, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi , Proc. ACM IMCOM2014 , 2014

The spread of cloud computing has increased the necessity of accumulating large amounts of data and performing high-speed data processing. Because strict consistency is not necessarily required for such large amount of data that cloud computing stores, a distributed Key Value Stores (KVS) is considered suitable for their data storage, based on an eventual consistency paradigm. In order to provide services such as SNS, mining and statistical processing of these data is indispensable. However because general distributed KVS systems are not designed for processing, these data must be transferred to distributed file systems such as HDFS, which enables data processing. The transfer cost issue has occurred in this case. To find a solution for this issue, we propose a method that performs high-speed data processing directly on a distributed KVS. In this paper, we extend the Apache Cassandra database, a distributed KVS that handles large amounts of data, to enable data affinity-based parallel processing. The parallel data processing mechanism runs the local processing on the stored values at each data node that stores the values, and it then returns only the results of the processing as an answer to a request. From the evaluation experiments, the proposed method is shown to be faster than the typically used Cassandra approach. In addition, even if the writing process is performed in the background while processing the data, the processing efficiency is appropriate for specific loads. The experimental results show that the data processing can be performed during the process of writing at approximately 10 Mbyte/sec if there are eight data nodes in the experiment environment.

A Scalable and Distributed Electrical Power Monitoring System Utilizing Cloud Computing [abst]
Ryousei Takano, Hidemoto Nakada, Toshiyuki Shimizu, Tomohiro Kudoh , The 8th International Conference on Ubiquitous Information Technologies and Applications , 2013

We propose a scalable and distributed electrical power monitoring system utilizing cloud computing. This system collects power usage at measurement points geographically distributed over different locations, stores data on the cloud and provides a single unified view of power usage through a simple REST API. A system with 620 measurement points covering a server room and a clean room has been successfully installed at our campus, and we have operated it since the third quarter of 2011 to charge electricity bills and evaluate the power efficiency of data processing middleware. We have demonstrated that the proposed system can be smoothly scaled out based on the needs. This result provides an insight that cloud computing makes a power monitoring system elastic and cost-effective.

A WAN-optimized Live Storage Migration Mechanism Toward Virtual Machine Evacuation Upon Severe Disasters [abst]
Takahiro Hirofuchi, Mauricio Tsugawa, Hidemoto Nakada, Tomohiro Kudoh, Satoshi Itoh , IEICE Transactions on Information and Systems, E96D 巻 12 号 , pp. 2663 - 2674 , 2013

Wide-area VM migration is a technology with potential to aid IT services recovery since it can be used to evacuate virtualized servers to safe locations upon a critical disaster. However, the amount of data involved in a wide-area VM migration is substantially larger compared to VM migrations within LAN due to the need to transfer virtualized storage in addition to memory and CPU states. This increase of data makes it challenging to relocate VMs under a limited time window with electrical power. In this paper, we propose a mechanism to improve live storage migration across WAN. The key idea is to reduce the amount of data to be transferred by proactively caching virtual disk blocks to a backup site during regular VM operation. As a result of pre-cached disk blocks, the proposed mechanism can dramatically reduce the amount of data and consequently the time required to live migrate the entire VM state. The mechanism was evaluated using a prototype implementation under diﬀerent workloads and network conditions, and we conﬁrmed that it dramatically reduces the time to complete a VM live migration. By using the proposed mechanism, it is possible to relocate a VM from Japan to the United States in just under 40 seconds. This relocation would otherwise take over 1500 seconds, demonstrating that the proposed mechanism was able to reduce the migration time by 97.5%.

Cooperative VM Migration: a Symbiotic Virtualization Mechanism by Leveraging the Guest OS Knowledge [abst]
Ryousei Takano, Hidemoto Nakada, Takahiro Hirofuchi, Yoshio Tanaka, Tomohiro Kudoh , IEICE Transactions on Information and Systems, E96D 巻 12 号 , pp. 2675-2683 , 2013

A virtual machine~(VM) migration is useful for improving flexibility and maintainability in cloud computing environments. However, VM monitor~(VMM)-bypass I/O technologies, including PCI passthrough and SR-IOV, in which the overhead of I/O virtualization can be significantly reduced, make VM migration impossible. This paper proposes a novel and practical mechanism, called Symbiotic Virtualization~(SymVirt), for enabling migration and checkpoint/restart on a virtualized cluster with VMM-bypass I/O devices, without the virtualization overhead during normal operations. SymVirt allows a VMM to cooperate with a message passing layer on the guest OS, then it realizes VM-level migration and checkpoint/restart by using a combination of a user-level dynamic device configuration and coordination of distributed VMMs. We have implemented the proposed mechanism on top of QEMU/KVM and the Open MPI system. All PCI devices, including Infiniband, Ethernet, and Myrinet, are supported without implementing specific para-virtualized drivers; and it is not necessary to modify either of the MPI runtime and applications. Using the proposed mechanism, we demonstrate reactive and proactive FT mechanisms on a virtualized Infiniband cluster. We have confirmed the effectiveness using both a memory intensive micro benchmark and the NAS parallel benchmark.

Power Efficient Virtual Machine Packing for Green Datacenter [abst]
Satoshi Takahashi, Atsuko Takefusa, Maiko Shigeno, Hidemoto Nakada, Tomohiro Kudoh, Akiko Yoshise , International Journal of Next-Generation Computing, Vol.4 No. 2 , pp. 162-181 , 2013

Cloud computing is now considered to be a new computing paradigm to provide scalable Infrastructure, Platform and Software as a Service via the Internet. While, the diffusion of Cloud computing is expected to cause an explosive increase in power consumption for IT resources in data centers. Virtual Machine(VM)-based flexible capacity management is an effective scheme to reduce total power consumption in the data centers. However, there remain the following issues, trade-off between power-saving and user experience, decision on VM packing plans within a feasible calculation time, and collision avoidance for multiple VM live migration processes. In order to resolve these issues, we propose two VM packing algorithms, a matching-based (MBA) and a greedy-type heuristic (GREEDY). MBA enables to decide an optimal plan in polynomial time, while GREEDY is an aggressive packing approach faster than MBA. We investigate the basic performance and the feasibility of proposed algorithms under both artificial and realistic simulation scenarios, respectively. The basic performance experiments show that the algorithms reduce total power consumption by between 18% and 50%, and MBA makes suitable VM packing plans within a feasible calculation time. The feasibility experiments employ two power consumption models, one is the linear model and the other is piecewise linear model. In the linear model, the feasibility experiments show that the reduction ratio of total power consumption observed with MBA is smaller than that of GREEDY, but the performance degradation of MBA is less than that of GREEDY. In the piecewise-linear model, the feasibility experiments show that MBA investigates more reducing power consumption than GREEDY. The performance degradation of MBA is also less than GREEDY.

A Distributed Application Execution System for an Infrastructure with Dynamically Configured Networks [abst]
Ryousei Takano, Hidemoto Nakada, Atsuko Takefusa, Tomohiro Kudoh , International Workshop on Network Infrastructure Services as part of IEEE Cloud Computing 2012 , 2012

We have been developing a middleware suite called GridARS that enables co-allocation of computing and network resources from multiple administration sites. In such middleware, it is important to provide each user application with a slice which is a set of dynamically allocated resources distributed across sites. However, there are several issues in constructing such a slice automatically. We design and implement an application execution system that provides each application with a slice, that mimics a conventional computing cluster system over the dynamically allocated resources. From the demonstration of the proposed system on an emulated wide area network environment, we confirmed that: first, the proposed system can fully automate resource allocation, slice construction, application invocation, and resource monitoring, in coordination with GridARS. Second, the proposed system can setup a slice quickly, even if the allocated resources are widely distributed and their communication latencies are high.

Virtual Machine Packing Algorithms for Lower Power Consumption [abst]
Satoshi Takahashi, Atsuko Takefusa, Maiko Shigeno, Hidemoto Nakada, Tomohiro Kudoh, Akiko Yoshise , IEEE CloudCom 2012 , 2012

Cloud computing is now considered to be a new computing paradigm to provide scalable Infrastructure, Platform and Software as a Service via the Internet. While, the diffusion of Cloud computing is expected to cause an explosive increase in power consumption for IT resources in data centers. Virtual Machine(VM)-based flexible capacity management is an effective scheme to reduce total power consumption in the data centers. However, there remain the following issues, tradeoff between power-saving and user experience, decision on VM packing plans within a feasible calculation time, and collision avoidance for multiple VM live migration processes. In order to resolve these issues, we propose two VM packing algorithms, a matching-based (MBA) and a greedy-type heuristic (GREEDY). MBA enables to decide an optimal plan in polynomial time, while GREEDY is an aggressive packing approach faster than MBA. We investigate basic performance and the feasibility of the proposed algorithms under both artificial and realistic simulation scenarios, respectively. In the feasibility experiments, we use an actual trace data set from the 648-node T2K-Tsukuba supercomputer. The basic performance experiments show that the algorithms reduce total power consumption by between 18% and 50%, and MBA makes suitable VM packing plans within a feasible calculation time. The feasibility experiments show that the proposed algorithms are feasible to make packing plans for an actual supercomputer, and GREEDY has the advantage in power consumption, but MBA shows the better performance in user experience.

Cooperative VM Migration for a Virtualized HPC Cluster with VMM-Bypass I/O devices [abst]
Ryousei Takano, Hidemoto Nakada, Takahiro Hirofuchi, Yoshio Tanaka, Tomohiro Kudoh , Proceedings of IEEE International Conference on eScience (eScience 2012) , 2012

An HPC cloud, a ﬂexible and robust cloud computing service specially dedicated to high performance computing, is a promising future e-Science platform. In cloud computing, virtualization is widely used to achieve ﬂexibility and security. Virtualization makes migration or checkpoint/restart of computing elements (virtual machines) easy, and such features are useful for realizing fault tolerance and server consolidations. However, in widely used virtualization schemes, I/O devices are also virtualized, and thus I/O performance is severely degraded. To cope with this problem, VMM-bypass I/O technologies, including PCI passthrough and SR-IOV, in which the I/O overhead can be signiﬁcantly reduced, have been introduced. However, such VMM-bypass I/O technologies make it impossible to migrate or checkpoint/restart virtual machines, since virtual machines are directly attached to hardware devices. This paper proposes a novel and practical mechanism, called Symbiotic Virtualization (SymVirt), for enabling migration and checkpoint/restart on a virtualized cluster with VMM-bypass I/O devices, without the virtualization overhead during normal operations. SymVirt allows a VMM to cooperate with a message passing layer on the guest OS, then it realizes VM-level migration and checkpoint/restart by using a combination of a PCI hotplug and coordination of distributed VMMs. We have implemented the proposed mechanism on top of QEMU/KVM and the Open MPI system. All PCI devices, including Inﬁniband and Myrinet, are supported without implementing speciﬁc para-virtualized drivers; and it is not necessary to modify either of the MPI runtime and applications. Using the proposed mechanism, we demonstrate reactive and proactive FT mechanisms on a virtualized Inﬁniband cluster. We have conﬁrmed the effectiveness using both a memory intensive micro benchmark and the NAS parallel benchmark. Moreover, we also show that postcopy live migration enables us to reduce the down time of an application as the memory footprint increases.

On the Use of Virtualization Technologies to Support Uninterrupted IT Services [abst]
Mauricio Tsugawa, Renato Figueiredo, Jose Fortes, Takahiro Hirofuchi, Hidemoto Nakada, Ryousei Takano , IEEE International Conference on Communications 2012 , 2012

Virtualized IT infrastructures combined with virtual machine migration technologies have a potential to support IT services that are resilient to partial physical infrastructure failures caused by extreme events. This paper experimentally evaluates the migration of multiple VMs across long geographical distances – an activity that is required to move virtualized IT systems from a disaster site to a safe location. Taking into account the resource availability parameters observed after the Great East Japan Earthquake, experimental results show that if (1) service downtime in the order of minutes is acceptable, (2) VMs can be kept with small storage footprint, and (3) power and network are available for tens of minutes, it is possible to migrate tens of VMs from damaged sites to a very distant stable location.

Kagemusha: A Guest-Transparent Mobile IPv6 Mechanism for Wide-Area Live VM Migration [Paper] [abst]
Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, Satoshi Sekiguchi , 3rd International Workshop on Cloud Management (CloudMan 2012) , 2012

We are developing a wide-area live migration mechanism that allows dynamic load balancing of virtual machines (VMs) among datacenters. We consider that Mobile IPv6 (MIPv6) is a promising technology to support transparent network reachability when VMs migrate to foreign networks. Existing MIPv6 mecha- nisms, however, are not suitable for VM migrations; real-world IaaS datacenters require guest-transparent and exible tunneling mechanisms, which are not provided by existing MIPv6 programs. In this paper, we propose a guest-transparent MIPv6 tunneling mechanism (Kagemusha), which performs Client MIPv6 signaling and tunneling on a host operating system. No MIPv6 program is required to be installed into a guest operating system. The proposed system is fully compatible with existing home agents (HAs). It basically works with any virtual machine monitors. Through experiments, we confirmed that our prototype system successfully established MIPv6 tunnels with HAs, and its performance overhead was negligible for normal use cases. We also confirrmed that the prototype system successfully worked with live migrations; the downtime of migration increased only by several hundred milliseconds.

Reactive Cloud: Consolidating Virtual Machines with Postcopy Live Migration [abst]
Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, Satoshi Sekiguchi , IPSJ Transaction Computing System (ACS) Vol.5 No.2 , pp. 86-98 , 2012

Dynamic consolidation of virtual machines (VMs) through live migration is a promising technology for IaaS datacenters. VMs are dynamically packed onto fewer server nodes, thereby eliminating excessive power consumption. Existing studies on VM consolidation, however, are based on precopy live migration, which requires dozens of seconds to switch the execution hosts of VMs. It is difficult to optimize VM locations quickly on sudden load changes, resulting in serious violations of VM performance criteria. In this paper, we propose an advanced VM consolidation system exploiting postcopy live migration, which greatly alleviates performance degradation. VM locations are reactively optimized in response to ever-changing resource usage. Sudden overloading of server nodes are promptly resolved by quickly switching the execution hosts of VMs. We have developed a prototype of our consolidation system and evaluated its feasibility through experiments. We confirmed that our consolidation system achieved a higher degree of performance assurance than using precopy migration. Our micro benchmark program, designed for the metric of performance assurance, showed that performance degradation was only 12% or less, even for memory-intensive workloads, which was less than half the level of using precopy live migration. The SPECweb benchmark showed that performance degradation was approximately 10%, which was greatly alleviated from the case of using precopy live migration (21%).

Transparent collaboration of GridRPC middleware using the OGF standardized GridRPC data management API [abst]
Yves Caniou, Gael le Mahec, Hidemoto Nakada, Eddy Caron , International Symposium on Grids and Clouds (ISGC) , 2012

In september 2011, the Open Grid Forum standardized the document "Data Management API within the GridRPC" [1] which discribes an optional API that extends the GridRPC standard [2]. Used in a GridRPC middleware, it provides a minimal set of functions to handle a large set of data operations among which: movements, replications, migrations, data prefetch and persistency. In this paper, we present a basic implementation of the API that we have integrated in two different middleware, respectively DIET [3] and NINF [4]. We have conducted several experiments, showing very high benefits that a Grid user can expect 1) in terms of resource usage compared to the current GridRPC context since useless transfers are avoided; 2) in terms of reducing the completion time of an application to obtain results the soonest (data can be prefetched and replicated, hence letting calculus to be submitted really soon in a workflow analysis in addition to the possible overlap between computations and communications); 3) in terms of code portability, since we show with these examples that at last the same GridRPC code can be compiled and executed within two different GridRPC middleware which implements the GridRPC data management API; 4) finally we thus obtain middleware interoperability without any explicit glue as generally done like in [5]: we show as a proof of concept that resources dispatched across different administrative domains can be used altogether without the underlying distributed data management systems having any knowledge of the workflow and/or computing resources: computational servers of DIET and NINF transparently collaborate to the same calculus by sharing GridRPC data.

Standardized Data Management in GridRPC Environments [abst]
Yves Caniou, Gael le Mahec, Eddy Caron, Hidemoto Nakada , 6th International Conference on Computer Sciences and Convergence Information Technology , pp. 501-508 , 2011

This paper details an extension to the GridRPC API that responds both to the data management needs of distributed applications and to middleware interoperability. It provides a minimal set of functions to handle a large set of data operations: movements, replications, migrations, persistency and data prefetch, which can be used by clients or workflow systems for example. We trust that this API covers all data operations required in GridRPC distributed environments.

GridARS: A Grid Advanced Resource Management System Framework for Intercloud [abst]
Atsuko Takefusa, Hidemoto Nakada, Ryousei Takano, Tomohiro Kudoh, Yoshio Tanaka , CloudCom 2011, NetCloud workshop , pp. 705-710 , 2011

Intercloud is a promising technology for data intensive applications. However, an important issue for Intercloud applications is orchestration of various virtualized and performance-assured resources, not only computers, but also network and storage, provided from multiple domains. We have been developing an advance reservation-based resource management framework, called Grid ARS, which can integrate heterogeneous resources and construct a performance-assured virtual infrastructure over Intercloud environment. Grid ARS provides four services that address resource management, resource allocation planning, provisioning and monitoring of the constructed virtual infrastructure. Grid ARS has been developed using common Web services technologies and standards. In this paper, we present overview of Grid ARS and its service components and describe Grid ARS demonstration challenges, demonstration at GLIF2010 and SC10 and OGF NSI interoperation in 2011.

Reactive Consolidation of Virtual Machines Enabled by Postcopy Live Migration [abst]
Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, Satoshi Sekiguchi , VTDC11 - The 5th International Workshop on Virtualization Technologies in Distributed Computing , pp. 11-18 , 2011

Dynamic consolidation of virtual machines (VMs) through live migration is a promising technology for IaaS datacenters. VMs are dynamically packed onto fewer server nodes, thereby eliminating excessive power consumption. Existing studies on VM consolidation, however, are based on precopy live migration, which requires dozens of seconds to switch the execution hosts of VMs. It is difficult to optimize VM locations quickly on sudden load changes, resulting in serious violations of VM performance criteria. In this paper, we propose an advanced VM consolidation system exploiting postcopy live migration, which greatly alleviates performance degradation. VM locations are reactively optimized in response to ever-changing resource usage. Sudden overloading of server nodes are promptly resolved by quickly switching the execution hosts of VMs. We have developed a prototype of our consolidation system and evaluated its feasibility through experiments. Our results show that our consolidation system achieved a higher degree of performance assurance than using precopy migration. Performance degradation is 12% or less, even for memory-intensive workloads, which is less than half the level using precopy migration.

SSS: An Implementation of Key-value Store based MapReduce Framework [Paper] [abst]
Hirotaka Ogawa, Hidemoto Nakada, Ryousei Takano, Tomohiro Kudoh , Proceedings of 2nd International Conference on Cloud Computing Technology and Science , pp. 745-761 , 2010

MapReduce has been very successful in implementing large-scale data-intensive applications. Because of its simple programming model, MapReduce has also begun being utilized as a programming tool for more general distributed and parallel applications, e.g., HPC applications. However, its applicability is limited due to relatively inefficient runtime performance and hence insufficient support for flexible workflow. In particular, the performance problem is not negligible in iterative MapReduce applications. On the other hand, today, HPC community is going to be able to utilize very fast and energy-efficient Solid State Drives (SSDs) with 10 Gbit/sec-class read/write performance. This fact leads us to the possibility to develop ``High-Performance MapReduce'', so called. From this perspective, we have been developing a new MapReduce framework called ``SSS'' based on distributed key-value store (KVS). In this paper, we first discuss the limitations of existing MapReduce implementations and present the design and implementation of SSS. Although our implementation of SSS is still in a prototype stage, we conduct two benchmarks for comparing the performance of SSS and Hadoop. The results indicate that SSS performs 1-10 times faster than Hadoop.

Grid Network Service - Web Services Interface Version 2, Achieving Scalable Reservation of Network Resources Across Multiple Network Domains [abst]
Yukio Tsukishima, Michiaki Hayashi, Tomohiro Kudoh, Akira Hirano, Takahiro Miyamoto, Atsuko Takefusa, Atsushi Taniguchi, Shuichi Okamoto, Hidemoto Nakada, Yasunori Sameshima, Hideaki Tanaka, Fumihiro Okazaki, Masahiko Jinno , IEICE Transaction on Communications, vol. E93-B, no 10 , pp. 2696-2705 , 2010

Platforms of hosting services are expected to provide a virtual private computing infrastructure with a guaranteed level of performance according to each reservation request sent from a client. To expand the performance of the computing infrastructure for reservation requests, the platforms are required to reserve, coordinate, and control globally distributed computing and network resources across multiple domains. This paper proposes Grid Network Service - Web Services Interface version 2 (GNS-WSI2). GNS-WSI2 is a resource-reservation messaging protocol that establishes a client-server relationship. A server allocates available network resources over its own domain according to each reservation request from a client. GNS-WSI2 has the capability to reserve network resources rapidly and reliably over multiple network domains. This paper also presents demonstration results of the feasibility of GNS-WSI2 in terms of the scalable reservation of network resources over multiple network domains in a transpacific testbed. In the demonstration, two computing infrastructures over multiple network domains are dynamically provided for scientific computing and remotevisualization applications, respectively. On the provided infrastructures, these applications are executed. The feasibility is successfully shown through the demonstration.

Eliminating Datacenter Idle Power with Dynamic and Intelligent VM Relocation [abst]
Takahiro Hirofuchi, Hidemoto Nakada, Hirotaka Ogawa, Satoshi Itoh, Satoshi Sekiguchi , Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living (Proceedings of International Symposium on Distributed Computing and Artificial Intelligence 2009) , pp. 645-648 , 2010

We are developing an advanced IaaS (Infrastructure-as-a-Service) datacenter management system that dynamically minimizes running physical servers depending on resource utilization. The management system periodically monitors the loading of a datacenter, and dynamically repacks virtual machines (VMs) into optimal physical servers. Live migration of VMs and the standby mode of physical servers are automatically orchestrated by a genetic algorithm (GA) engine. A preliminary experiment showed that our first prototype system correctly worked for a proof-of-concept datacenter.

Enabling Instantaneous Relocation of Virtual Machines with a Lightweight VMM Extension [abst]
Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, Satoshi Sekiguchi , The 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing (CCGrid2010) , pp. 73-83 , 2010

We are developing an efficient resource management system with aggressive virtual machine (VM) relocation among physical nodes in a data center. Existing live migration technology, however, requires a long time to change the execution host of a VM, it is difficult to optimize VM packing on physical nodes dynamically, corresponding to ever-changing resource usage. In this paper, we propose an advanced live migration mechanism enabling instantaneous relocation of VMs. To minimize the time needed for switching the execution host, memory pages are transferred after a VM resumes at a destination host. A special character device driver allows transparent memory page retrievals from a source host for the running VM at the destination. In comparison with related work, the proposed mechanism supports guest operating systems without any modifications to them (i.e, no special device drivers and programs are needed in VMs). It is implemented as a lightweight extension to KVM (Kernel-based Virtual Machine Monitor). It is not required to modify critical parts of the VMM code. Experiments were conducted using the SPECweb2005 benchmark. A running VM with heavily-loaded web servers was successfully relocated to a destination within one second. Temporal performance degradation after relocation was resolved by means of a precaching mechanism for memory pages. In addition, for memory intensive workloads, our migration mechanism moved all the states of a VM faster than existing migration technology.

An Advance Reservation-based Co-Allocation Algorithm for Distributed Computers and Network Bandwidth on QoS-guaranteed Grids [abst]
Atsuko Takefusa, Hidemoto Nakada, Tomohiro Kudoh, Yoshio Tanaka , 15th Workshop on Job Scheduling Strategies for Parallel Processing , 2010

Co-allocation of performance-guaranteed computing and network resources provided by several administrative domains is one of the key issues for constructing a QoS-guaranteed Grid. We propose an advance reservation-based co-allocation algorithm for both computing and network resources on a QoS- guaranteed Grid, modeled as an integer programming (IP) problem. The goal of our algorithm is to create reservation plans satisfying user resource requirements as an on-line service. Also the algorithm takes co-allocation options for user and resource administrator issues into consideration. We evaluate the proposed al- gorithm with extensive simulation, in terms of both functionality and practicality. The results show: The algorithm enables efficient co-allocation of both computing and network resources provided by multiple domains, and can reflect reservation options for resource administrators issues as a first step. The calculation times needed for selecting resources using an IP solver are acceptable for an on-line service.

Contribution of Virtualization Technologies to Reducing Data Center Power Consumption
Satoshi Itoh, Hidemoto Nakada, Takahiro Hirofuchi, Hirotaka Ogawa, Satoshi Sekiguchi , The 6th International Symposium on Environmentally Conscious Design and Inverse Manufacturing (EcoDesign2009) , pp. 6p , 2009
A Live Storage Migration Mechanism over WAN and its Performance Evaluation [abst]
Takahiro Hirofuchi, Hidemoto Nakada, Hirotaka Ogawa, Satoshi Itoh, Satoshi Sekiguchi , roceedings of the 3rd International Workshop on Virtualization Technologies in Distributed Computing (VTDC2009) , pp. 67-74 , 2009

Live migration of virtual machines is a key technology for the next generation of IaaS cloud services, contributing to dynamic portability and mobility of VM-based services among datacenters. The practical use of live migration, however, is still limited inside a single datacenter. In WAN environments, network latencies cause inevitable I/O performance degradation of remotely-shared storage between source and destination sites; which is required to continue disk access of VMs before/after live migration. In our previous work, we proposed a transparent, relocatable I/O mechanism for VM migration, which enables VM disk images to be completely migrated to remote nodes without any modification of virtual machine monitors. In this paper, we present detailed performance evaluation of the proposed system, emulating a realistic WAN environment between remote datacenters. Experiments showed the proposed system achieved feasible I/O performance for various workloads including I/O intensive applications. Its background copy mechanism efficiently prefetches not-yet-cached blocks by exploiting the available bandwidth of WAN, thereby minimizing temporary perfor- mance degradation of the migrating VM system.

Toward Virtual Machine Packing Optimization based on Genetic Algorithm [abst]
Hidemoto Nakada, Takahiro Hirofuchi, Hirotaka Ogawa, Satoshi Itoh , Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living (Proceedings of International Symposium on Distributed Computing and Artificial Intelligence 2009) , pp. 651-654 , 2009

To enable efficient resource provisioning in HaaS (Hardware as a Service) cloud systems, virtual machine packing, which migrate vir- tual machines to minimize running real node, is essential. The virtual machine packing problem is a multi-objective optimization problem with several parameters and weights on parameters change dynamically sub- ject to cloud provider preference. We propose to employ Genetic Algorithm (GA) method, that is one of the meta-heuristics. We implemented a prototype Virtual Machine packing optimization mechanism on Grivon, which is a virtual cluster management system we have been developing. The preliminary evaluation implied the GA method is promising for the problem.

Design of a Domain Authorization-based Hierarchical Distributed Resource Monitoring System in cooperation with Resource Reservation [abst]
Atsuko Takefusa, Hidemoto Nakada, Seiya Yanagita, Fumihiro Okazaki, Tomohiro Kudoh, Yoshio Tanaka , Proc. HPC Asia 2009 , pp. 77-84 , 2009

Grid and Network provisioning technology has enabled the construction of high-quality virtual computing infrastructures spanning several administrative organizations. However, it is still difficult for users to monitor the usage of distributed and various resources managed by multiple domains. We propose an authorization-based hierarchical distributed resource monitoring system called DMS that gathers information based on resource reservation, and filters information with the policies specified by the administrators using XACML, which is a standard authorization model and a policy description language. DMS co-works with the GridARS co-allocation framework to retrieve resource reservation information and adopts web services technologies and an extension of a standard data representation set. To confirm feasibility of the DMS system, we describe monitoring strategies for reserved computing and network resources in Collectors and we have developed a WSRF-based DMS prototype, which enables authorization by XACML. The experiments using the prototype system show: (1) Even when DMS employs a large number of policies, the overhead of the XACML authorization decision process is negligible, since that of WSRF/GSI is more dominant in the total processing time, and (2) the benefits of parallel information aggregation from multiple domains make the retrieval latency acceptable.

Speculative Checkpointing: Exploiting Temporal Affinity of Memory Operations [abst]
Satoshi Matsuoka, Ikuhei Yamagata, Hideyuki Jitsumoto, Hidemoto Nakada , Proc. HPC Asia 2009 , pp. 390-397 , 2009

In large scale parallel systems, storing memory images with checkpointing will involve massive amounts of concentrated I/O from many nodes, resulting in considerable execution overhead. For user-level checkpointing, overhead reduction usually involves both spatial, i.e., reducing the amount of checkpoint data, and temporal, i.e., spreading out I/by checkpointing data as soon as their values become fixed. However, for system-level check-pointing, while being generic and effortless for the end-user, most efforts have focused on simple methods for spatial reductions only. Instead, we propose speculative checkpointing, which is an attempt to exploit temporal memory operation affinity in system-level check-pointing. We demonstrate that speculative check-pointing can be implemented as a simple extension of incremental checkpointing, well-known check-pointing optimization algorithm for spatial reduction. Although shown to be useful and effective, the overall effectiveness of speculative checkpointing is greatly affected by the last-write heuristics of pages, and as such it is difficult to determine the theoretical upper bound of the effectiveness of speculative checkpointing in practical applications. In order to analyze this, we construct a "checkpointing oracle simulator" that allows post-mortem analysis of maximal temporal reduction in checkpoint time given an application. The benchmarks show that speculative checkpointing can reduce up to 32% of checkpointing time in NAS parallel benchmarks.

The Proxy-based Design Pattern for Grid Middleware and its Application: Ninf-G5 [abst]
Hidemoto Nakada, Yoshio Tanaka, Satoshi Sekiguchi , Proc. HPC Asia 2009 , pp. 210-217 , 2009

This paper proposes a design pattern for composing grid middleware and introduces Grid RPC system Ninf-G implementation as an application of the pattern. There are not a few libraries and toolkits proposed to help composing applications and middleware on the grid. The most commonly used style for composing applications and middleware will be `monolithic'; i.e., the core logic are written using the provided API and directly linked with the libraries into a single binary. While this style is intuitive, there are several issues on this style; 1) Middleware/applications have to be written in the same language with the library, 2) When APIs of libraries update, the core portion of the middleware/applications have to be modified, 3) Mixed use of several libraries might cause serious name collision or dependency collision. We propose a proxy-based design pattern to compose grid middleware/applications, which is conducted from our years of experience with implementing Grid RPC system Ninf-G, where the each function of the library will be allocated separate process which communicate with the middleware/applications' core process with simple protocol. This design pattern allows the core process to be portable among several grid libraries/toolkits and to be written in arbitrary language. We introduce GridRPC Ninf-G version 5 implementation as an application of the design pattern. We compared it with the previous incarnation of the same system, which is implemented in monolithic way from two point of view; invocation latency and communication throughput. Contrary to the intuitive expectation, the result showed that the impact of the design pattern is negligible in terms of invocation latency and acceptable in terms of communication throughput.

A Live Storage Migration Mechanism over WAN for Relocatable Virtual Machine Services on Clouds [abst]
Takahiro Hirofuchi, Hirotaka Ogawa, Hidemoto Nakada, Satoshi Itoh, Satoshi Sekiguchi , Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, International Workshop on Cloud Computing (Cloud 2009) , pp. 460-465 , 2009

IaaS (Infrastructure-as-a-Service) is an emerging concept of cloud computing, which allows users to obtain hardware resources from virtualized datacenters. Although many commercial IaaS clouds have recently been launched, dynamic virtual machine (VM) migration is not possible among service providers; users are locked into a particular provider, and cannot transparently relocate their VMs to another one for the best cost-effectiveness. In this paper, we propose an advanced storage access mechanism that strongly supports live VM migration over WAN. It rapidly relocates VM disks between source and destination sites with the minimum impact on I/O performance. The proposed mechanism addresses I/O consistency of virtual disks before/after migration, which is the major issue regarding wide-area live migration. The proposed mechanism works as a storage server of a block-level storage I/O protocol (e.g., iSCSI and NBD). Two key techniques (on-demand fetching and background copying) move on-line virtual disks among remote sites, transparently and efficiently. Our prototype system works perfectly for Xen and KVM without any modification to them. Experiments showed the prototype system also worked successfully for an emulated WAN environment.

A Multi-Site Virtual Cluster System for Wide Area Networks [abst]
Takahiro Hirofuchi, Takeshi Yokoi, Tadashi Ebara, Yusuke Tanimura, Hirotaka Ogawa, Hidemoto Nakada, Yoshio Tanaka, Satoshi Sekiguchi , Proceedings of the First USENIX Workshop in Large-Scale Computing, , 2008

A virtual cluster is a promising technology for reducing management costs and improving capacity utilization in datacenters and computer centers. However, re- cent cluster virtualization systems do not have the maximum scalability and flexibility required, due to limited hardware resources at one site. Therefore, we are now developing an advanced cluster management system for multi-site virtual clusters; which provides a virtual cluster composed of distributed computer resources over wide area networks. This system has great advan- tages over other cluster management systems designed only for single-site resources; users can create a cluster of virtual machines from local and remote physical clusters in a scalable manner, and dynamically change the number of cluster nodes on demand, seamlessly. In our system, a multi-site cluster achieves a monolithic system view of cluster nodes to enable existing appli- cations to be deployed quickly and managed flexibly, just as in physical clusters. In this paper, we propose an advanced cluster virtualization mechanism composed of two key techniques. An L2 network extension of virtual machine networks allows transparent deployment over networks for distributed virtual cluster nodes, and a transparent package caching mechanism greatly opti- mizes data transfers in virtual cluster deployment over network latencies. Experimental results show multi-site virtual clusters have sufficient feasibility in WAN envi- ronments and promise great scalability for a large-scale number of virtual nodes.

Intelligent data staging with overlapped execution of grid applications [abst]
Yuuya Machida, Shinichiro Takizawa, Hidemoto Nakada, Satoshi Matsuoka , FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF GRID COMPUTING THEORY METHODS AND APPLICATIONS Vol.24 Issue5 , pp. 425 - 433 , 2008

Existing data grid scheduling systems handle huge data I/O via replica location services coupled with simple staging and decoupled from scheduling of computing tasks. However, when the application/workflow scales, we observe considerable degradations in performance, compared to processing within a tightly-coupled cluster. For example, when numerous nodes access the same set of files simultaneously, extreme performance degradation occurs even if replicas are used, due to bottlenecks that show in the infrastructure. Instead of resorting to expensive solutions such as parallel file systems, we propose tightly coupling replica and data transfer management with computation scheduling for alleviating such situations. In particular, we propose three techniques: (1) data-staging requests aggregation and O(1) replication across multiple nodes using a multireplication framework, (2) replica-centric scheduling, which reuses previously used data for minimizing staging time and (3) overlapped execution of data staging and compute bound tasks. Early benchmark results implemented in our prototype Condor-like grid scheduling system demonstrate that the techniques are quite effective in eliminating much of the overhead in data transfers and achieving 100% of CPU utilization.

The Design and Implementation of a Virtual Cluster Management System [Paper] [Slides] [abst]
Hidemoto Nakada, Takeshi Yokoi, Tadashi Ebara, Yusuke Tanimura, Hirotaka Ogawa, Satoshi Sekiguchi , Proc. of 1st IEEE/IFIP International Workshop on End-to-end Virtualization and Grid Management , pp. 61-71 , 2007

To fully utilize resources in computer center, virtualization techniques are getting popular and several systems are proposed for this purpose. However, they just provide set of virtualized nodes, not the 'virtual clusters'; i.e., they are not able to install and configure middlewares and tools that makes 'set of nodes' into 'cluster'. Another problem is that they just virtualize nodes, leaving storage resources and networks, which are equivalently essential for clusters, un-virtualized. We propose a virtual cluster management system which virtualizes compute resources, as well as disk storage and network, and install and setup softwares that are essential for cluster operation, using Rocks, a cluster provisioning system. We virtualize storage with iSCSI and network with tagged VLAN.

G-lambda and EnLIGHTened: Wrapped In Middleware Co-allocating Compute and Network Resources Across Japan and the US [abst]
Steven R. Thorpe, Lina Battestilli, Gigi Karmous-Edwards, Andrei Hutanu, Jon MacLaren, Joe Mambretti, John H. Moore, Kamaraju Syam Sundar, Yufeng Xin, Atsuko Takefusa, Michiaki Hayashi, Akira Hirano, Shuichi Okamoto, Tomohiro Kudoh, Takahiro Miyamoto, Yukio Tsukishima, Tomohiro Otani, Hidemoto Nakada, Hideaki Tanaka, Atsushi Taniguchi, Yasunori Sameshima, Masahiko Jinno , Proc. First International Conference on Networks for Grid Applications (GridNets) , pp. 8p , 2007

This paper describes innovative architectures and techniques for reserving and coordinating highly distributed resources, a capability required for many large scale applications. In the fall of 2006, Japan's G-lambda research team and the United States' EnLIGHTened Computing research team used these innovations to achieve the world's first inter-domain coordination of resource managers for in-advance reservation of network bandwidth and compute resources between and among both the US and Japan. The compute and network resource managers had different interfaces and were independently developed. Automated interoperability among the resources in both countries was enabled through various Grid middleware components. In this paper, we describe the middleware components, testbeds, results, and lessons learned.

Job invocation interoperability between NAREGI Middleware Beta and gLite [Paper] [Slides] [abst]
Hidemoto Nakada, Hitoshi Sato, Kazushige Saga, Masayuki Hatanaka, Yuji Saeki, Satoshi Matsuoka , Proc. of HPC Asia 2007 , pp. 151-158 , 2007

As grid middleware stacks mature, the importance of inter-operation among them is getting more significant. There is a community group called GIN (Grid Interoperation Now) in the OGF (Open Grid Forum), a standardization body for grid related technologies, which aims to establish interoperation among several grid middlewares. We performed experiments on inter-operation between our NAREGI Middleware beta and EGEE gLite, as a contribution to the group. For the experiments, we implemented several modules to allow information exchange and mutual job submissions. As the results of the experiment, we confirmed the followings: 1) The security layer, such as certificates and virtual organization management, imposes no fundamental difficulties despite the subtle diffrences in the use of proxy certificates, 2) While information services differs substantially, the resource information can be translated to allow effective information exchange via schema translation between GLUE and CIM, 3) Jobs can be mutually submitted based on the exchanged information, despite the differences in job description languages and interfaces (JSDL vs. JDL).

Autonomically-Adapting Master-Worker Programming Framework for Multi-Layered Grid-of-Clusters [Paper] [Slides] [abst]
Hitoshi Aoki, Hidemoto Nakada, Kouji Tanaka, Satoshi Matsuoka , Proc. of HPC Asia 2007 , pp. 3-10 , 2007

Past work on “programming for the grid” for compute-intensive applications have largely focused on two ex-tremes, either loosely-coupled, cycle-scavenging, desktop grid environments such as BOINC or Condor vs. tightly-coupled metacomputing systems such as MPICH-G2 or GridMPI. Neither is really appropriate for class of applications that are middle-tier, e.g., fine-grained branch-and-bound master-worker applications where the running time of individual jobs may range from less than tenth of a second to few minutes, and communication intervals being further fine-grained.Our new grid program- mingmiddleware and framework, Jojo2, allows efficient programming of fine-grained, hierarchicalmaster-worker applications on a grid-of-clusters environment. During programming, much of the complexities associated with hierarchy and changes in the underlying resources are hidden away or isolated, and the user need not be aware of physical configuration of the resources. Even during execution, new compute resources can be explicitly be added via batch submissions, and are subsequently automatically detected and incorporated in the system. Evaluation of Jojo2 on real master-worker applications proved very positive, both on TSUBAME, a supercomputing cluster with 10,000 nodes, as well as a nationwide testbed involving more than 800 CPUs and a number of 100-node class clusters. In both cases, applications on Jojo2 not only scaled very well, but autonomously adapted to bulk changes in the order of hundreds of underlying resources, both resources being added as well as those going away.

An Advance Reservation-based Computation Resource Manager for Global Scheduling [Paper] [Slides] [abst]
Hidemoto Nakada, Atsuko Takefusa, Katsuhiko Ookubo, Tomohiro Kudoh, Yoshio Tanaka, Satoshi Sekiguchi , GCA2007 Proceedings of the 3rd International Workshop on Grid Computing and Applications , pp. 3-14 , 2007

Advance Reservation is one possible way to enable resource co-allocation on the Grid. This method requires all the resources to have advance reservation capability as well as coordination protocol support. We employed 2-phased commit protocol as a coordination protocol, which is common in the distributed transaction area, and implemented an Advance Reservation Manager called {\bf PluS}. PluS works with existing local queuing managers, such as TORQUE or Grid Engine, and provides users advance reservation capability. To provide the capability, there are two implementation methods; 1) completely replaces the scheduling module of the queuing manger, 2) represents reservation as a queue and controls the queues using external interface. We designed and implemented a reservation manager with both way, and evaluated them. We found that the former has smaller overhead and allows arbitrary scheduling policy, while the latter is much easier to implement withacceptable response time.

GridARS: An Advance Reservation-based Grid Co-allocation Framework for Distributed Computing and Network Resources [abst]
Atsuko Takefusa, Hidemoto Nakada, Tomohiro Kudoh, Yoshio Tanaka, Satoshi Sekiguchi , Proc. 13th Workshop on Job Scheduling Strategies for Parallel Processing (LNCS 4942), Seattle , pp. 152-168 , 2007

For high performance parallel computing on actual Grids, one of the important issues is to co-allocate the distributed resources that are managed by various local schedulers with advance reservation. To address the issue, we proposed and developed the GridARS resource co-allocation framework, and a general advance reservation protocol that uses WSRF/GSI and a two-phased commit (2PC) protocol to enable a generic and secure advance reservation process based on distributed transactions, and provides the interface module for various existing resource schedulers. To confirm the effectiveness of GridARS, we describe the performance of a simultaneous reservation process and a case study of GridARS grid co-allocation over transpacific computing and network resources. Our experiments showed that: 1) the GridARS simultaneous 2PC reservation process is scalable and practical and 2) GridARS can co-allocate distributed resources managed by various local schedulers stably.

Design and Implementation of Distributed Task Sequencing on GridRPC [Paper] [abst]
Yusuke Tanimura, Hidemoto Nakada, Yoshio Tanaka, Satoshi Sekiguchi , Proc. of 2006 IEEE International Conference on Computer and Information Technology , pp. 6p , 2006

In the framework of GridRPC, a new function that allows direct data transfer between RPC servers is implemented for ef cient execution of a Task Sequencing job in a grid environment. In Task Sequencing, RPC requires dependency between input and output parameters, which means output of a previous RPC becomes the input of the next RPC. In this study, the direct transfer of data is implemented using the grid lesystem without destroying the GridRPC programming model and without changing very many parts of the existing Ninf-G implementation. Our Task Sequencing API library analyzes RPC arguments to detect intermediate data after task submissions, and reports the information to GridRPC servers so that the intermediate data is created on the grid lesystem. Through our performance evaluation on LAN and on the Japan-US grid environment, it was verified that the function achieved performance improvement in distributed Task Sequencing.

Design and Implementation of a Local Scheduling System with Advance Reservation for Co-allocation on the Grid [Paper] [Slides] [abst]
Hidemoto Nakada, Atsuko Takefusa, Katsuhiko Ookubo, Makoto Kishimoto, Tomohiro Kudoh, Yoshio Tanaka, Satoshi Sekiguchi , Proc. of 2006 IEEE International Conference on Computer and Information Technology , pp. 6p , 2006

While advance reservation is an essential capability for co-allocating several resources on Grid environments, it is not obvious how it can co-exist with priority-based First Come First Served scheduling, that is widely used as local scheduling policy today. To investigate this problem, we 1) developed a scheduling API in Java for TORQUE, a variant of OpenPBS, that enables users to implement their own schedulers and replace the original scheduling module with them, 2) implemented a prototype scheduler module that has advance reservation capability with the API. We also provide an external interface for the reservation capability based on WSRF to enable co-allocation of resources over the Grid. Using this interface with the job submission module from Globus toolkit 4, users can make reservation for resources and submit jobs over the Grid.

Multi-Replication with Intelligent Staging in ata-Intensive Grid Applications [Paper] [abst]
Yuya Machida, Shinichiro Takizawa, Hidemoto Nakada, Satoshi Matsuoka , The 7th IEEE/ACM International Conference on Grid Computing , pp. 88-95 , 2006

Existing data grid scheduling systems handle huge data I/O via replica location services coupled with simple staging,decoupled from scheduling of computing tasks. However,when the application/workflow scales, we observe considerabledegradations in performance, compared to processing withina tightly-coupled cluster. For example, when numerous nodesaccess the same set of files simultaneously, major performancedegradation occurs even if replicas are used, due to bottlenecksthat manifest in the infrastructure. Instead of resorting toexpensive solutions such as parallel file systems, we proposealleviating the situation by tightly coupling replica and datatransfer management with computation scheduling. In particularwe propose three techniques: (1) dynamic aggregation and O(1)replication of data-staging requests across multiple nodes usinga multi-replication framework, (2) replica-centric scheduling ?data re-use and time-to-replication as compute scheduling metricson the grid and (3) overlapped execution of data staging and compute bound tasks. Early benchmark results implemented inour prototype Condor-like grid scheduling system demonstratethat the techniques are quite effective in eliminating much of theoverhead in data transfers in many cases.

G-lambda: Coordination of a Grid Scheduler and Lambda Path Service over GMPLS [abst]
Atsuko Takefusa, Michiaki Hayashi, Naohide Nagatsu, Hidemoto Nakada, Tomohiro Kudoh, Takahiro Miyamoto, Tomohiro Otani, Hideaki Tanaka, Masatoshi Suzuki, Yasunori Sameshima, Wataru Imajuku, Masahiko Jinno, Yoshihiro Takigawa, Shuichi Okamoto, Yoshio Tanaka, Satoshi Sekiguchi , FUTURE GENERATION COMPUTER SYSTEMS 22-2006 , pp. 868-875 , 2006

At iGrid2005, we conducted a live demonstration where our Grid scheduling system co-allocated computing and network resources with advance reservation through Web services interfaces using the Grid Resource Scheduler (GRS), the Network Resource Management System (NRM), which is capable of GMPLS network resource management, and a GMPLS-based network test-bed, for the first time. The goal of the G-lambda project is to define a standard Web services interface (GNS-WSI) between GRS and NRM that is acceptable for both application service providers and commercial network operators, and which can be used as a tool for realizing new and emerging commercial services.

Managing and Controlling GMPLS Network Resources for Grid Applications [abst]
Michiaki Hayashi, Takahiro Miyamoto, Tomohiro Otani, Hideaki Tanaka, Atsuko Takefusa, Hidemoto Nakada, Tomohiro Kudoh, Naohide Nagatsu, Yasunori Sameshima, Shuichi Okamoto , Proc. Optical Fiber Communication Conference, 2006 , 2006

Inter-working between GMPLS network and grid computing application through Web-services interface is demonstrated for the first time. Lambda LSP-based network resource virtualization and scheduling techniques successfully achieves nation-wide grid computing environment with advance reservation operation.

Speculative Checkpointing [abst]
Ikuhei Yamagata, Satoshi Matsuoka, Hidemoto Nakada , Dependable Software Workshop '06 , pp. 9p , 2006

In large scale parallel systems, storing memory images with checkpointing will involve massiveamounts of concentrated I/O from many nodes,resulting in considerable execution overhead. Foruser-level checkpointing, overhead reduction usuallyinvolves both spatial, i.e., reducing the amountof checkpoint data, and temporal, i.e., spreadingout I/O by checkpointing data as soon as their valuesbecome fixed. However, for system-level checkpointing,while being generic and effortless for theend-user, most efforts have focused on simple methodsfor spatial reductions only. Instead, we proposespeculative checkpointing, which is an attempt toexploit temporal reduction in system-level checkpointing.We demonstrate that speculative checkpointingcan be implemented as a simple extensionof incremental checkpointing, a well-known checkpointingoptimization algorithm for spatial reduction.Although shown to be useful and effective,the overall effectiveness of speculative checkpointingis greatly affected by the last-write heuristics ofpages, and as such it is difficult to determine thetheoretical upper bound of the effectiveness of speculativecheckpointing in practical applications. Inorder to analyze this, we construct a checkpointing oracle simulator that allows post-mortem analysisof maximal temporal reduction in checkpoint timegiven an application. The benchmarks show thatspeculative checkpointing can reduce up to 32% ofcheckpointing time in NAS parallel benchmarks.

Design and implementation of Flexible, Robust and Efficient Grid-enabled Hybrid QM/MD Simulation [abst]
Yoshio Tanaka, Hiroshi Takemiya, Hidemoto Nakada, Satoshi Sekiguchi , Computational Methods in Science and Technology,Volume 12 (1) 2006, 79-87 , pp. 79-87 , 2006

This paper describes the implementation of Grid-enabled hybrid Quantum Mechanics/Classical Molecular Dynamics (QM/MD) Simulation. The Grid-enabled QM/MD simulation is capable of (1) dynamic resource allocation and migration, (2) automatic recovery from faults, and (3) managing large number of CPUs in geographically distributed sites. In the implementation, both GridRPC and MPI are used to complement each other, that is, GridRPC allows dynamic resource allocation and migration and automatic recovery from faults while MPI provides high-performance parallel processing on each cluster. We ran the hybrid QM/MD simulation on an international Grid testbed in the Asia Pacific Region for about 52 days. The experimental results indicated that the hybrid QM/MD simulation could (1) adapt to the dynamic behavior of the simulation and can change the number of CPUs and the number of clusters, (2) adapt to unstable Grid infrastructure and recovers from faults automatically, and (3) manage hundreds to thousands of CPUs on distributed locations, and (4) survive for several weeks without interrupting manual operation. This paper reports on the details of the implementation, strategies for long-run, and experimental results.

Primary Study of A Task Farming API over The GridRPC Framework [Paper] [Slides] [abst]
Yusuke Tanimura, Hidemoto Nakada, Yoshio Tanaka, Satoshi Sekiguchi , Eighth International Conference on High-Performance Computing in Asia-Pacific Region , pp. 339-345 , 2005

In this paper, a middleware suite, which provides a Task Farming API, is studied in use over the GridRPC standard, in order to reduce the complexity of developing task parallel applications for the grid. APIs are proposed and higher functionality in task scheduling and fault tolerance is implemented in the middleware, based on our past experiences with the Ninf-G. Through our study, it is revealed that the Argument Array API needs to provide a means to copy arguments for duplicated task assignment. Timing of data transfer in the non-blocking RPC and a method to retrieve execution information for each RPC are expected to be standardized in the GridRPC. By resolving these three issues in the GridRPC, our Task Farming API library, meeting application requirements, can be fully realized on multiple GridRPC systems, paving the way for other higher functional API libraries to be designed and implemented.

Design and Implementation of Condor-UNICORE Bridge [Paper] [Slides] [abst]
Hidemoto Nakada, Jaime Frey, Motohiro Yamada, Yasuyoshi Itou, Yasumasa Nakano, Satoshi Matsuoka , Eighth International Conference on High-Performance Computing in Asia-Pacific Region , pp. 307-314 , 2005

In this paper, we describe the design and implementation of a generic grid interface for Condor. Though Condor has intefaces for specific grid systems, such as Globus Toolkit 2 GRAM, it is not easy to add new interfaces for other grid systems, since it requires some code modification inside the Condor. With our new interface design, supporting a new grid system can be established without any code modification in the Condor core itself. We also implemented a bridge for the UNICORE system and validated that our approach is effective.

A Scalable Multi-Replication Framework for Data Grid [Paper] [abst]
Shinichiro Takizawa, Yasuhito Takamiya, Hidemoto Nakada, Satoshi Matsuoka , Proceedings of the 2005 International Symposium on Applications and the Internet (SAINT 2005 Workshops) , pp. 310-315 , 2005

Existing replica services on the Grid we know to date assumes point-to-point communication and file transfer protocol. As such, when hundreds to thousands of hosts on the Grid access a single dataset simultaneously, bottlenecks in networks and/or the data servers will hinder performance significantly. Instead, our replication framework couples efficient, multicast techniques with a replica catalog that automatically detects simultaneous access to the replica by multiple nodes. As a prototype, we have designed and built a portable, XML-based replica location service accounting for such parallel transfer requests, and coupled it with a O(1) bulk file transfer system Dolly+[6]. The benchmarks show that the system is scalable and effective in reducing replication costs significantly in cluster-based replication scenarios.

The Design and implementation of a Fault-Tolerant RPC system: Ninf-C [Paper] [abst]
Hidemoto Nakada, Yoshio Tanaka, Satoshi Matsuoka, Satoshi Sekiguchi , Proceedings of HPC ASIA 2004 , pp. 9-18 , 2004

We describe the design and implementation of a fault tolerant GridRPC system, Ninf-C, designed for easy programming of large-scale master-worker programs that take from few days to few months for its execution in a Grid environment. Ninf-C employs Condor, developed at University of Wisconsin, as the underlying middleware supporting remote file transmission and checkpointing for system-wide robustness for application users on the Grid. Ninf-C layers all the GridRPC communication and task parallel programming features on top of Condor in a non-trivial fashion, assuming that the entire program is structured in a masterworker style?in fact, older Ninf master-worker programs can be run directly or trivially ported to Ninf-C. In contrast to the original Ninf, Ninf-C exploits and extends Condor features extensively for robustness and transparency, such as 1) checkpointing and stateful recovery of the master process, 2) the master and workers mutually communicating using (remote) files, not IP sockets, and 3) automated throttling of parallel GridRPC calls; and in contrast to using Condor directly, programmers can set up complex dynamicworkflow as well as master-worker parallel structure with almost no learning curve involved. To prove the robustness of the system, we performed an experiment on a heterogeneous cluster that consists of x86 and SPARC CPUs, and ran a simple but long-running master-worker program with staged rebooting of multiple nodes to simulate some serious fault situations. The program execution finished normally avoiding all the fault scenarios, demonstrating the robustness of Ninf-C.

GridSpeed: A Web-based Grid Portal Generation Server [Paper] [abst]
Toyotaro Suzumura, Hidemoto Nakada, Satoshi Matsuoka, Henri Casanova , Proceedings of HPC ASIA 2004 , pp. 26-33 , 2004

GridSpeed is a grid portal hosting server that automatically generates and publishes a customized web interface to the grid for applications, with minimal effort required from the user. Users need only to specify information regarding their application using simple GridSpeed web forms. With GridSpeed, users need not make any modi cations to their applications nor write any glue code to publish the application on the web, not requiring any knowledge of Perl, JSP (Java Server Pages) or Java Servlets. Moreover, the portal generated by GridSpeed provides an application frontend as well as a set of fundamental portal services such as an information service, monitoring service, data management, single sign-on, and so forth. GridSpeed publishes a set of portals as Grid services themselves generated by the system that is sharable, searchable, and accessible from others interested in using the application. This feature facilitates the reuse of application portals for speci c application domains, as well as increases the number of available Grid applications accessible on the web. This paper describes an overview and architecture of the GridSpeed system, and evaluates the system for two real-world scienti c applications: BLAST and MCell.

Parallelization of Phylogenetic Tree Inference using Grid Technologies [Paper] [Slides] [abst]
Yo Yamamoto, Hidemoto Nakada, Hidetoshi Shimodaira, Satoshi Matsuoka , LSGrid 2004 , pp. 103-116 , 2004

The maximum likelihood method is considered as one of the most reliable methods for phylogenetic tree inference. However, as the number of species increases, the approach quickly loses its applicability due to explosive exponential number of trees that need to be considered. An earlier work by one of the authors [3] demonstrated that, by decomposing the trees into fragments called splits, and calculating the individual likelihood of each (small) split and combining them would result in a very close approximation of the true maximum likelihood value, as well as achieving significant reduction in computational cost. However, the cost was still significant for a practical number of species that need to be considered. To solve this problem, we further extend the algorithm so that it could be effectively parallelized in a Grid environment using Grid middleware such as Ninf and Jojo, and also applied combinatorial optimization techniques. Combined, we achieved over 64 times speedup over our previous results in a testbed of 16 nodes, with favorable speedup characteristics.

A Java-based Programming Environment for the Grid: Jojo [Slides] [abst]
Hidemoto Nakada, Satoshi Matsuoka, Satoshi Sekiguchi , Proceedings of CCGrid 2004 , pp. 51-58 , 2004

Despite recent developments in higher-level middleware for the Grid supporting high level of ease-of-programming, hurdles for widespread adoption of Grids remain high, due to (1) assumption of peer-to-peer connectivity of all Grid nodes, as well as (2) lack of scalable programming and deployment support. We propose a Java-based programming environment for a hierarchically organized Grid named Jojo, that allow seamless utilization of privately addressed clusters. Jojo provides several features, including secure private remote invocation using Globus GRAM and ssh/rsh to privately addressed nodes in clusters, intuitive message passing API suitable for overlapped execution using multiple threads, and automatic user/system program staging. Using Jojo, users can easily construct and execute parallel distributed applications on the Grid. We show the design and implementation of its programming API, a working example, as well as preliminary performance evaluation results that prove the effectiveness of hierarchal execution.

Design, implementation and performance evaluation of GridRPC programming middleware for a large-scale computational Grid [Paper] [abst]
Yoshio Tanaka, Hiroshi Takemiya, Hidemoto Nakada, Satoshi Sekiguchi , Proceeding of 5th IEEE/ACM International Workshop on Grid Computing. , pp. 298-305 , 2004

This paper reports on the design, implementation and performance evaluation of a suite of GridRPC programming middleware called Ninf-G Version 2 (Ninf-G2). Ninf-G2 is a reference implementation of the GridRPC API, a proposed GGF standard. Ninf-G2 has been designed so that it provides 1) high performance in a large-scale computational Grid, 2) the rich functionalities which are required to adapt to compensate for the heterogeneity and unreliability of a Grid environment, and 3) an API which supports easy development and execution of Grid applications. Ninf-G2 is implemented to work with basic Grid services, such as GSI, GRAM, and MDS in the Globus Toolkit version 2. The performance of Ninf-G2 was evaluated using a weather forecasting system which was developed using Ninf-G2. The experimental results indicate that high performance can be attained even in relatively fine-grained task-parallel applications on hundreds of processors in a Grid environment.

Autonomous Configuration of Grid Monitoring Systems [abst]
Ken'ichiro Shirose, Satoshi Matsuoka, Hidemoto Nakada, Hirotaka Ogawa , Proceedings of the 2004 International Symposium on Applications and the Internet (SAINT 2004 Workshops) , pp. 651-657 , 2004

The problem with practical, large-scale deployment of Grid monitoring system is that, it takes considerable management cost and skills to maintain the level of quality required by production usage, since the monitoring system will be fundamentally be distributed, need to be running continuously, and will itself likely be affected by the various faults and dynamic reconfigurations of the Grid itself. Although their automated management would be desirable, there are several difficulties, distributed faults and reconfigurations, component interdependencies, and scaling to maintain performance while minimizing probing effect. Given our goal to develop a generalized autonomous management framework for Grid monitoring, we have built a prototype, on top of NWS, featuring automatic configuration of its ``clique'' groups as well as coping with single-node faults without user intervention. An experimental deployment on the Tokyo Institute of Technology's Campus Grid (The Titech Grid) consisting of over 15 sites and 800 processors has shown the system to be robust in handling faults and reconfigurations, automatically deriving an ideal clique configuration for the head login nodes of each PC cluster in less than two minutes.

Ninf-G: A Reference Implementation of RPC-based Programming Middleware for Grid Computing [abst]
Yoshio Tanaka, Hidemoto Nakada, Satoshi Sekiguchi, Toyotaro Suzumura, Satoshi Matsuoka , Journal of Grid Computing, Vol. 1, No. 1 , pp. 41-51 , 2003

GridRPC, which is an RPC mechanism tailored for the Grid, is an attractive programming model for Grid computing. This paper reports on the design and implementation of a GridRPC programming system called Ninf-G. Ninf-G is a reference implementation of the GridRPC API which has been proposed for standardization at the Global Grid Forum. In this paper, we describe the design, implementations and typical usage of Ninf-G. A preliminary performance evaluation in both WAN and LAN environments is also reported. Implemented on top of the Globus Toolkit, Ninf-G provides a simple and easy programming interface based on standard Grid protocols and the API for Grid Computing. The overhead of remote procedure calls in Ninf-G is acceptable in both WAN and LAN environments.

Evaluation of the inter-cluster data transfer on Grid environment [abst]
Shoji Ogura, Satoshi Matsuoka, Hidemoto Nakada , Proceedings oft the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2003), , pp. 374-381 , 2003

High-performance peer-to-peer transfer between clusters will be fundamental technology base for various Grid middleware, such as large-scale data transfer in DataGrid settings, or collective communication in Grid-wide MPIs. There, two major factors are involved: on one hand network pipes with large RTT × bandwidth typically become data-starved, resulting in bandwidth loss; on the other hand when multiple nodes on the clusters attempt simultaneous transfer, the network pipe could become saturated, resulting in packet loss which again may result in bandwidth degradation in large RTT × bandwidth networks. By dynamically and automatically adjusting transfer parameters between the two clusters, such as the number of network nodes, number of socket stripes, we could achieve optimal bandwidth even when the network is under heavy contention. In order to arrive at a proper performance model for automated adjustment, we have conducted several simulations by which we have discovered that such automatic tuning would beneficial, but the ideal number of network pipes does not exactly match the simple transfer model of traditional peer-to-peer settings between single nodes.

Overview of GridRPC: A Remote Procedure Call API for Grid Computing [abst]
Keith Seymour, Hidemoto Nakada, Satoshi Matsuoka, Jack Dongarra, Craig Lee, Henri Casanova , Grid Computing - Grid 2002, LNCS 2536 , pp. 274-278 , 2002

This paper discusses preliminary work on standardizing and implementing a remote procedure calll (RPC) mechanism for grid computing. The GridRPC API is designed to address the lack of a standardized, portable, and simple programming interface. Our initial work on GridRPC shows that client access to existing grid computing systems such as NetSolve and Ninf can be unified via a common API, a task that has proven to be problematic in the past.

The Ninf Portal: An Automatic Generation Tool for the Grid Portals [abst]
Toyotaro Suzumura, Hidemoto Nakada, Masayuki Saito, Satoshi Matsuoka, Yoshio Tanaka, Satoshi Sekiguchi , Proceeding of Java Grande 2002 , pp. 1-7 , 2002

As the Grid proliferates as the next-generation computing infrastructure, a user interface in the form of "Grid Portals" is becoming increasingly important, especially for computational scientists and engineers. Although several Grid Portal toolkits have been proposed, the portal developer still must build and deploy both the user interface and the application, which results in considerable programming efforts. We aim to ease this burden by generating the portal frontend (that constitutes of JSP and Java Servlets) from an XML document for the former, and a GridRPC system, Ninf-G for easily "gridifying" existing applications for the latter, and realizing their seamless integration. The resulting system, which we call the Ninf Portal, allowed concise description and easy deployment of a real Grid application with greatly small programming efforts.

Evaluating Web Services Based Implementations of GridRPC [abst]
Satoshi Shirasuna, Hidemoto Nakada, Satoshi Matsuoka, Satoshi Sekiguchi , Proceeding of HPDC11 , pp. 237-245 , 2002

GridRPC is a class of Grid middleware for scientific computing. Interoperability has been an important issue, because current GridRPC systems each employ its own protocol. Web services, where XML-based standards such as SOAP and WSDL are expected to see widespread use, could be the medium of interoperability; however, it is not clear if 1) XML-based schemas have sufficient expressive power for GridRPC, and 2) whether performance could be made sufficient. Our experiments indicate that the use of such technologies are more promising than previously reported. Although a naive implementation of SOAP-based GridRPC has severe performance overhead, application of a series of optimizations improves performance. However, encoding of various features of GridRPC proved to be somewhat difficult due to WSDL limitations. The results show that GridRPC systems can be based on Web technologies, but there needs to be work to extend WSDL specifications, possibly impacting OGSA-based Grid services directions.

Implementation of Portable Software DSM in Java [abst]
Yukihiko Sohda, Hidemoto Nakada, Hirotaka Ogawa, Satoshi Matsuoka , Proc. of JavaGrande 2001 , pp. 163-162 , 2001

Rapid commoditization of advanced hardware and progress of networking technology is now making wide area high-performance computing a.k.a. the `Grid' Computing a reality. Since a Grid will consist of vastly heterogeneous sets of compute nodes, especially commodity clusters, some have articulated the use of Java as a suitable technology to satisfy portability across different machines. Since Java's natural model of parallelism is shared memory multithreading, one will have to support distributed shared memory (DSM) in a portable manner; however, none of the previous work on implementing Java on DSM has been a portable solution. Instead, we propose a software architecture whose goal is to achieve portability of DSM implementations across different commodity clustering platforms, while restricting the programming model somewhat, and implemented a prototype system, JDSM. Benchmark results show that the current implementation on Java incurs increased memory coherency maintenance cost compared to C-based DSMs, thus limiting scalability to some degree, and we are currently working on a solution to alleviate this cost.

A Jini-based Computing Portal System [abst]
Toyotaro Suzumura, Satoshi Matsuoka, Hidemoto Nakada , SC '01: Proceedings of the 2001 ACM/IEEE conference on Supercomputing , 2001

JiPANG(A Jini-based Portal Augmenting Grids) is a portal system and a toolkit which provides uniform access interface layer to a variety of Grid systems, and is built on top of Jini distributed object technology. JiPANG performs uniform higher-level management of the computing services and resources being managed by individual Grid systems such as Ninf, NetSolve, Globus, etc. In order to give the user a uniform interface to the Grids JiPANG provides a set of simple Java APIs called the JiPANG Toolkits, and furthermore, allows the user to interact with Grid systems, again in a uniform way, using the JiPANG Browser application. With JiPANG, users need not install any client packages beforehand to interact with Grid systems, nor be concerned about updating to the latest version. Such uniform, transparent services available in a ubiquitous manner we believe is essential for the success of Grid as a viable computing platform for the next generation.

Design issues of Network Enabled Server Systems for the Grid [abst]
Satoshi Matsuoka, Hidemoto Nakada, Mitsuhisa Sato, Satoshi Sekiguchi , Grid Computing – GRID 2000, Springer-Verlag, LNCS 1971 , pp. 4-17 , 2001

Network Enabled Server is considered to be a good candidate as a viable Grid middleware, offering an easy-to-use programming model. This paper clarifies design issues of Network Enabled Server systems and discusses possible choices, and their implications, namely those concerning connection methodology, protocol command representation, security methods, etc. Based on the issues, we have designed and implemented new Ninf system v.2.0. For each design decision we describe the rationale and the details of the implementation as dictated by the choices. We hope that the paper serves as a design guideline for future NES systems for the Grid.

Performance Evaluation Model for Scheduling in Global Computing Systems [abst]
Kento Aida, Atsuko Takefusa, Hidemoto Nakada, Satoshi Matsuoka, Satoshi Sekiguchi, Umpei Nagashima , International Journal of High-Performance Computing Applications, vol.20, Vol. 14, No. 3 , pp. 268-279 , 2000

Striking progress of network technology is enabling high-performance global computing, in which computational and data resources in a wide area network (WAN) are transparently employed to solve large-scale problems. Several high-performance global computing systems, such as Ninf, NetSolve, RCS, Legion and Globus have already been proposed. Each of these systems proposes to effectively achieve high-performance with some efficient scheduling scheme, whereby a scheduler selects a set of appropriate computing resources that solve the client's computational problem. This paper proposes a performance evaluation model for effective scheduling in global computing systems. The proposed model represents a global computing system by a queueing network, in which servers and networks are represented by queueing systems. Verification of the proposed model and evaluation of scheduling schemes on the model showed that the model could simulate behavior of an actual global computing system and scheduling on the system effectively.

Performance Evaluation of a Firewall-compliant Globus-based Wide-area Cluster System [abst]
Yoshio Tanaka, Mototaka Hirano, Mitsuhisa Sato, Hidemoto Nakada, Satoshi Sekiguchi , 9th IEEE International Symposium on High Performance Distributed Computing (HPDC 2000) , pp. 121-128 , 2000

In this paper, we present a performance evaluation of a wide-area cluster system based on a firewall-enabled Globus metacomputing toolkit. In order to establish communication links beyond the firewall, we have designed and implemented a resource manager called RMF (Resource Manager beyond the Firewall) and the Nexus Proxy, which relays TCP communication links beyond the firewall. In order to extend the Globus Metacomputing Toolkit to the firewall-enabled toolkit, we have built the Nexus Proxy into the Globus toolkit. We have built a firewall-enabled Globus-based wide-area cluster system in Japan and run some benchmarks on it. In this paper, we report various performance results such as the communication bandwidth and latencies obtained as well as application performance involving a tree search problem. In a wide-area environment, the communication latency through the Nexus Proxy is approximately six times larger when compared to that of direct communications. As message size increases however, the communication overhead caused by the Nexus Proxy can be negligible. We have developed a tree search problem using MPICH-G. We used a self-scheduling algorithm, which is considered to be suitable for a distributed heterogeneous metacomputing environment since it performs dynamic load balancing with low overhead. The performance results indicate that the communication overhead caused by the Nexus Proxy is not a severe problem in metacomputing environments.

Are Global Computing Systems Useful? - Comparison of Client-Server Global Computing Systems Ninf, NetSolve versus CORBA [abst]
Toyotaro Suzumura, Takayuki Nakagawa, Satoshi Matsuoka, Hidemoto Nakada, Satoshi Sekiguchi , Proc. of International Parallel and Distributed Processing Symposium , pp. 547-556 , 2000

Recent developments of global computing systems such as Ninf, NetSolve and Globus have opened up the opportunites for providing high-performance computing services over wide-area networks. However, most research focused on the individual architectural aspects of the system, or application deployment examples, instead of the necessary charactersistics such systems should intrinsically satisfy, nor how such systems relate with each other. Our comparative study performs deployment of example applications of network-based libraries using Ninf, NetSolve, and CORBA systems. There, we discover that dedicated systems for global computing such as Ninf and NetSolve have management, progammability, and in does not suffer performance disadvantages over more generic distributed computing capabilities provided by CORBA. Such results indicate the advantage of dedicated global computing systems over general systems, stemming further basic research is necessary across multiple systems to identify the ideal software architectures for global computing.

NetCFD: a Ninf CFD component for Global Computing, and its Java applet GUI [abst]
Mitsuhisa Sato, Kazuhiro Kusano, Hidemoto Nakada, Satoshi Sekiguchi, Satoshi Matsuoka , Proc. of HPC Asia 2000 , pp. 501-506 , 2000

Ninf is a middleware for building a global computing system in wide area network environments. We designed and implemented a Ninf computational component, netCFD for CFD (Computational Fluid Dynamics). The Ninf Remote Procedure Call (RPC) provides an interface to a parallel CFD program running on any high performance platforms. The netCFD turns high performance platforms such as supercomputers and clusters into valuable components for use in global computing. Our experiment shows that the overhead of a remote netCFD computation for a typical application was about 10\% comparing with its conventional local execution. The netCFD applet GUI which is loaded in a web browser allows a remote user to control and visualize the CFD computation results interactively.

Overview of a Performance Evaluation System for Global Computing Scheduling Algorithms [abst]
Atsuko Takefusa, Satoshi Matsuoka, Hidemoto Nakada, Kento Aida, Umpei Nagashima , 8th IEEE International Symposium on High Performance Distributed Computing (HPDC8) , pp. 97-104 , 1999

While there have been several proposals of high performance global computing systems, scheduling schemes for the systems have not been well investigated. The reason is difficulties of evaluation by large-scale benchmarks with reproducible results. Our Bricks performance evaluation system would allow analysis and comparison of various scheduling schemes on a typical high-performance global computing setting. Bricks can simulate various behaviors of global computing systems, especially the behavior of networks and resource scheduling algorithms. Moreover, Bricks is componentalized such that not only its constituents could be replaced to simulate various different system algorithms, but also allows incorporation of existing global computing components via its foreign interface. To test the validity of the latter characteristics, we incorporated the NWS system, which monitors and forecasts global computing systems behavior. Experiments were conducted by running NWS under a real environment versus the simulated environment given the observed parameters of the real environment. We observed that Bricks behaved in the same manner as the real environment, and NWS also behaved similarly, making quite comparative forecasts under both environments.

Resource Manager for Globus-based Wide-area Cluster Computing [abst]
Yoshio Tanaka, Mototaka Hirano, Mitsuhisa Sato, Hidemoto Nakada, Satoshi Sekiguchi , 1st IEEE International Workshop on Cluster Computing (IWCC'99) , pp. 237-244 , 1999

In this paper, we present a new type of Globus resource allocation manager (GRAM) called RMF (Resource Manager beyond the Firewall) for wide-area cluster computing. RMF manages computing resources such as cluster systems and enables utilization of them beyond the firewall in global computing environments. RMF consists of two basic modules, a remote job queuing system (Q system) and a resource allocator. The Q system is a remote job queuing system, which schedules jobs submitted from global computing sites. The resource allocator manages resources and allocates them for requested jobs. For communications beyond the firewall between job processes, we designed the Nexus Proxy, which relays TCP communication links beyond the firewall. RMF and the Nexus Proxy provide a global computing environment in which users can easily utilize such parallel systems as cluster systems and supercomputers beyond the firewall.

Design and Implementations of Ninf: towards a Global Computing Infrastructure [abst]
Hidemoto Nakada, Mitsuhisa Sato, Satoshi Sekiguchi , Future Generation Computing Systems, Metacomputing Issue, Vol.15, Issues 5-6 , pp. 649-658 , 1999

The world-wide computing infrastructure on the growing computer network technology is a leading technology to make a variety of information services accessible through the Internet for every users from the high performance computing users through many of personal computing users. The important feature of such services is location transparency; information can be obtained irrespective of time or location in virtually shared manner. In this article, we overview Ninf, an ongoing global network-wide computing infrastructure project which allows users to access computational resources including hardware, software and scientific data distributed across a wide area network. Preliminary performance result on measuring software and network overhead is shown, and that promises the future reality of world-wide network computing.

Ninf and PM: Communication Libraries for Global Computing and High-performance Cluster Computing [abst]
Mitsuhisa Sato, Hiroshi Tezuka, Atsushi Hori, Yutaka Ishikawa, Satoshi Sekiguchi, Hidemoto Nakada, Satoshi Matsuoka and Umpei Nagashima , Future Generation Computing Systems vol. 13 (1997/98), No. 4-5 , pp. 349-359 , 1998

This paper presents two advanced communication libraries, Ninf and PM. Ninf is an ongoing global network-wide computing infrastructure project which allows users to access computational resources including hardware, software and scientific data distributed across a wide area network. Computational resources are shared as Ninf remote libraries executable at a remote Ninf server. Users can build an application by calling the libraries with the Ninf Remote Procedure Call. In order to facilitate location transparency and network-wide parallelism, Ninf metaserver maintains global resource information regarding computational server and databases, allocating and scheduling coarse-grained computation for global load balancing. PM is a high performance communication library for workstation clusters connected with myrinet gigabit LAN card, which has a dedicated processor and on-board memory to handle a communication protocol. In order to obtain high performance communication and support a multi-user environment, we co-designed PM, an operating system realized by a daemon process, and the run-time routine for a programming language. Several unique features, e.g., network context switching and modified ACK/NACK flow control algorithm have been developed for PM. PM for the Suns has a speed of 20 micro seconds round trip for a user-level 8 bytes message and 38.6 Mbytes/second bandwidth for an 8 Kbytes message.

Utilizing the Metaserver Architecture in the Ninf Global Computing System [abst]
Hidemoto Nakada, Hiromitsu Takagi, Satoshi Matsuoka, Umpei Nagashima, Mitsuhisa Sato, Satoshi Sekiguchi , High-Performance Computing and Networking '98, LNCS 1401 , pp. 607-616 , 1998

Rapid increase in speed and availability of global-network is opening up the possibilities of globally-distributed supercomputing, including our Ninf system.Ninf is designed to utilize existing numerical libraries and resources on the Web. It also has global scheduling server called the Ninf Metaserver, which is responsible for scheduling and load-distribution of parallel tasks. The metaserver keeps track of computational servers, and gathers information needed for scheduling. Using the information, it balances loads over computational servers.We performed preliminary benchmarks using the metaserver. There, we observed that scheduling cost can be ignored and dynamic scheduling by the server gained better score than static-cyclic distribution.We also report on our collaborative efforts in bridging Ninf with NetSolve, a similar system being developed at Univ. of Tennessee/ORNL.

Ninflet: a Migratable Parallel Objects Framework using Java [abst]
Hiromitsu Takagi, Satoshi Matsuoka, Hidemoto Nakada, Satoshi Sekiguchi, Mitsuhisa Satoh, Umpei Nagashima , ACM 1998 Workshop on Java for High-Performance Network Computing , pp. 151-159 , 1998

Ninflet is a Java-based global computing system that builds on our experiences with the Ninf system which facilitated RPC-based computing of numerical tasks in a wide-area network. The goal of Ninflet is to become a new generation of concurrent object-oriented system which harness abundant idle computing powers, and also seamlessly integrate global as well as local network parallel computing. Ninflet is designed to make use of Java features to implement important features in global computing, such as resource allocation, inter- Ninflet communication, security, checkpointing, object migration, and easy server management via HTTP.

Multi-client LAN/WAN Performance Analysis of Ninf: a High-Performance Global Computing System [abst]
Atsuko Takefusa, Satoshi Matsuoka, Hirotaka Ogawa, Hidemoto Nakada, Hiromitsu Takagi, Mitsuhisa Sato, Satoshi Sekiguchi, Umpei Nagashima , SC '97: Proceedings of the 1997 ACM/IEEE conference on Supercomputing , 1997

Rapid increase in speed and availability of network of supercomputers is making high-performance global computing possible, including our Ninf system. However, critical issues regarding system performance characteristics in global computing have been little investigated, especially under multi-client, multi-site WAN settings. In order to investigate the feasibility of Ninf and similar systems, we conducted benchmarks under various LAN and WAN environments, and observed the following results: 1) Given sufficient communication bandwidth, Ninf performance quickly overtakes client local performance, 2) current supercomputers are sufficient platforms for supporting Ninf and similar systems in terms of performance and OS fault resiliency, 3) for a vector-parallel machine (Cray J90), employing optimized data-parallel library is a better choice compared to conventional task-parallel execution employed for non-numerical data servers, 4) computationally intensive tasks such as EP can readily be supported under the current Ninf infrastructure, and 5) for communication-intensive applications such as Linpack, server CPU utilization dominates LAN performance, while communication bandwidth dominates WAN performance, and furthermore, aggregate bandwidth could be sustained for multiple clients located at different Internet sites; as a result, distribution of multiple tasks to computing servers on different networks would be essential for achieving higher client-observed performance. Our results are not necessarily restricted to the Ninf system, but rather, would be applicable to other similar global computing systems.

Ninf: A Network based Information Library for a Global World-Wide Computing Infrastracture [abst]
Mitsuhisa Sato, Hidemoto Nakada, Satoshi Sekiguchi, Satoshi Matsuoka, Umpei Nagashima and Hiromitsu Takagi , HPCN'97 (LNCS-1225) , pp. 491-502 , 1997

Ninf is an ongoing global network-wide computing infrastructure project which allows users to access computational resources including hard-ware, software and scientific data distributed across a wide area network. Ninf is intended not only to exploit high performance in network parallel computing, but also to provide high quality numerical computation services and accesses to scientific database published by other researchers. Computational resources are shared as Ninf remote libraries executable at a remote Ninf server. Users can build an application by calling the libraries with the Ninf Remote Procedure Call, which is designed to provide a programming interface similar to conventional function calls in existing languages, and is tailored for scientific computation. In order to facilitate location transparency and network-wide parallelism, Ninf metaserver maintains global resource information regarding computational server and databases, allocating and scheduling coarse-grained computation for global load balancing. Ninf also interfaces with the WWW browsers for easy accessi-bility.

– Ninf – : Network base information library for globally high performance computing
Satoshi Sekiguchi, Mitsuhisa Sato, Hidemoto Nakada, Umpei Nagashima , Parallel Object-Oriented Methods and Applications (POOMA) , 1996
Stepwise Synthesis of Partial Specifications preserving Strong (Q1, Q2)-Equivalence [abst]
Yoshinao Isobe, Hidemoto Nakada, Yutaka Sato, Kazuhito Ohmaki , Concurrency Theory and Applications '96 , pp. 39-53 , 1996

In this paper we present a partial equality called strong $(\Omega_{1}, \Omega_{2})$ -equivalence between two partial specifications $SP_{1}$ and $SP_{2}$, denoted by $SP_{1}\Omega_{1^{\sim}}\Omega_{2}SP_{2}$, based on an extended bisimulation relation. The parameters $\Omega_{1}$ and $\Omega_{2}$ are the sets of available actions in $SP_{1}$ and $SP_{2}$, respectively. Furthermore we present a synthesis method of the two partial specifications $SP_{1}$ and $SP_{2}$ into a specification $SP_{12}$ such that $SP_{12}\Omega_{1}\cup\Omega_{2}\sim_{\Omega_{1}}SP_{1}$ and $SP_{12}\Omega_{1}\cup\Omega_{2}\sim_{\Omega_{2}}SP_{2}$. $SP_{12}$ is called the principal strong $(\Omega_{1}, \Omega_{2})$ -synthetic specification of $SP_{1}$ and $SP_{2}$, and it is denoted by $SP_{12} \simeq(SP_{1\Omega_{1}}\prod_{\Omega_{2}}SP_{2})$. For example, this synthesis method is used for stepwise refinement of partial specifications preserving strong $(\Omega_{1}, \Omega_{2})$ -equivalence.

System Integration of the Parallel Inference Engine PIE64
Takuya Araki, Yasuo Hidaka, Hidemoto Nakada, Hanpei Koike, Hidehiko Tanaka , FGCS'94 Workshop on Parallel Logic Programming , pp. 64-76 , 1994
Load Distribution System of PIE64
Satoshi Murakami, Hidemoto Nakada, Yasuo Hidaka, Hanpei Koike, Hidehiko Tanaka , FGCS'94 Workshop on Parallel Logic Programming , pp. 77-90 , 1994
A Fleng Compiler for PIE64
Hidemoto Nakada, Takuya Araki, Hanpei Koike, Hidehiko Tanaka , Parallel Architectures and CompilationTechniques (PACT '94) , pp. 257-266 , 1994