Multimodal Discourse and Oral Language Skills in Digital Portfolios for Learning Foreign Languages

Our world today is ruled by technology, a fact that directly influences the way students learn. In addition, as teachers, we have witnessed how this situation is projected in the field of pedagogy. New pedagogical models are required to conform to the new ways of learning of the so-called "net generation" (Tapscott, 2008)[1]. Also, the use of cyber tools for learning foreign languages not only means that teachers have a wide range of learning resources at their disposal, but it also involves the creation of new discourses and modes of communication in the teaching context. This article presents the results of an investigation that was conducted during a semester with two groups of students who worked with a digital portfolio for learning a foreign language as a part of their formal training and evaluation. Results show learner ́s multimodal discourse analysis to ascertain how the discursive production of SFL1 students in digital portfolios integrate oral language in multimodal communication and how it enhances the learning process of oral language skills.


Introduction
In approaching the discourse analysis productions of SFL students, the question of what kind of discourse we would want to examine was raised. Since in current communication practices students communicate with each other using social networks and digital media, it was decided that the approach to the analysis of language learners discursive productions would be based on their productions present in a digital medium. The importance of exploring what kind of discursive productions arise when there is an ubiquitous learning (Nicholas Burbules, Guirtz and Necuzzi, 2011) [2] in SFL was also taken into consideration. Given this reality, this paper describes the discursive production characteristics 1 Spanish as Foreign Language of SFL students through digital tools. The digital media with which students worked and in which the analysis will be based on are the digital portfolio (Mahara) and the digital poster (Glogster). On one hand, the use of a digital portfolio for teaching and learning SFL -instead of a paper portfoliowill improve the use and didactic application that one can achieve with a portfolio (Cassany, 2012: 261) [3], as it allows you to manipulate and store various physical media, it is more manageable, it puts the authors in contact and creates social networks. On the other hand, the sister tool 2.0 Glogster has also been considered as a tool that has some of the same functions as a digital portfolio for learning a foreign language although it is only an interactive poster.

Multimodal Communication
In approaching the study of the discursive production of SFL students in a cybernetic medium, it is essential to define and limit what type of communication occurs.
When discussing the communication that occurs in virtual spaces, a key concept is that of multimodal communication Kress, 2010) [4,5]. Beginning with the semiotic theories of communication, a multimodal approach will be followed to analyze learning evidences produced by students in their digital portfolio and digital poster. The multimodal communication contemplates the various resources that can be used within communication modes, such as: image, written text, the disposition of information, music, gestures, spoken language, moving images or 3D objects, among others. The mode is understood as the semiotic resources used to give meaning to a particular communicative situation. The modes can be carried out in more than one way of production, ie, it can be performed using more than one resource material and also sister tools of the production, for example: a camera, a tape recorder, a computer, the vocal apparatus, etc-(Kress & Van Leeuwen, 2001: 20-23) [6].
As multimodality is an intrinsic characteristic of the digital narrative blogs (Pujolà & Montmany, 2007: 10) [7], it is believed that it is also a characteristic of the digital portfolio. In both digital spaces, multimodality is visible in the use and interaction in several ways with the aim of creating a global meaning to respond to the author's communicative intent. The centrality of verbal language has been left behind, and from a multimodal perspective we rely on the idea that meaning is produced, distributed, interpreted and (re) availed through many modes of representation and communication (Kress & Van Leeuwen, 2001) [8]. We find ourselves, therefore, at a crossroads of various modes and medias, which will be carried out thanks to the existence of open source software (OSS) and their use in the teaching of foreign languages.
By working with a multimodal discourse, not only are the range of semiotic modes seen as important, but also the relationship and interaction that is established between them. The combination of distinct semiotic modes enriches the creation of meaning, producing what has been described as multimodal sincretism (Garcia Palomeque, 2012: 15) [9]. Thus, the interaction of different semiotic modes (O'Holloran, 2011) [10] in the multimodal phenomena makes it possible for semantic expansion to occur, owing to the integration between the modes, favoring, in this way, its communicative function.

The Digital Portfolio Mahara
Mahara is an open-source OS-web-based software that allows users to upload documents and information in various forms and arrangements. Each of the presentations of the documents and information is called "view". Through Mahara one can create and manage digital portfolios, while using social networks that permit its users to communicate and interact with each other.

The Digital Poster Glogster
Glogster is also an open source OS-web-based-software. It has great graphic appeal and vast educational possibilities. It permits the development of interactive posters, offering countless resources: funds, titles, text, text figures, frames, etc., and facilitates the insertion of images, text and videos, but does not allow the inclusion of code or Flash documents.

Objectives
Our goal is to observe -in both digital medias -the use of different modes of communication carried out by the students (audiovisual, images, text and signs) and to see how they integrate them to enhance the development of communicative competence in oral language.
The potential of oral language learning when done through multimodal communication will be explored. Is the use of multimodality a support / compensatory strategy for students' oral communication? Does the use of multimodality in communication improve oral language skills?
In the analysis of the data corpus, questions related to cohesive strategies of the oral text will not be explored. The objective of this research is not to analyze the functioning of the oral text discursive elements but rather to explore how the oral language of SFL learners is part of the multimodal interaction that occurs in the multimodal discourse in a digital portfolio and a digital poster.

Data collection
The study is based on the implementation of the use of the Mahara portfolio for a semester. The participating students belong to an institution -American College IES Abroad-in which learners are in a foreign language context, ie, are spending a semester studying and living in a language immersion situation in Barcelona. The total number of students who participated in the research was fifteen. The students were divided into two groups of seven and eight respectively.
Students are learners of the B1 level in Spanish. All of them have studied a minimum of two semestres in their universities in the United States. Furthermore, before starting the course in Barcelona they had to take a placement test on-line. The level in which they were palced on the course was based on the result of this test. On arrival in IES Abroad Barcelona, they were reassessed to make sure that they were in the appropiate level.
The digital portfolio based on our study will only consist of one view, with two phases: Phase 1: Students have to do a presentation of themselves in order to find a partner for a language exchange with Education students studying English at the University of Barcelona. For the presentation it is necessary to use the free software Glogster, which will allow them to integrate video, written text and images. Phase 2: Students have to explain what the Hispanic culture means for them using again Glogster.
Teachers who conducted the research used the digital portfolio as a tool for teaching / learning through which learners worked with oral language as a component of multimodal communication. In addition, the portfolio of each of the students was another form of course evaluation, with a weight of 10% for the final grade.
For the analysis of the data the two glogsters have been taken into account separately and also the composition the Mahara portfolio view.
For the analysis of the glogsters, six samples belonging to the first task and six samples belonging to the second task have been selected. The criteria followed for selection was based on the length of video that appeared, which had to exceed a minute in length. It was felt that in order to achieve a more rich and meaningful analysis of the audiovisual document, it was imperative that the document would last for at least one minute. For this reason, from the thirty samples of all the glogsters produced by the students (fifteen samples from the first and fifteen from the second), only the glogsters

Results
The results will be developed and analyzed, describing and listing the categories that have been established to carry out multimodal discourse analysis. In order to achive an accurate overview of the categories established for the analysis, some basic concepts will be discussed.

Typology of Semiotic Modes
To set the range of semiotic modes that appear in the Glogster and quantify how many different modes are used, the number of words, images and video length have been counted.
In a first more descriptive and purely observant analysis, it can be seen that all of the Glogsters 1 exceed 10 images from real images obtained from the Internet or the clip art of the glog design program itself. In addition, the videos attached by the students in the glog exceed the minimum time expectation, 1:05 min being the shortest and almost 3 minutes being the longest. Finally, the number of words going from 5 up to 30 words. The total number of words is 79, of images 68 and the duration of the videos is 10 minutes and 78 seconds. After this first test of perception, as previously stated, we see that there is no relationship between the amount of text and video length or the number of images. It seems, however, that there is a slight relationship or trend between the amount of text and the number of images. That is, that more images mean more text.
Through the analysis of the Glogsters 2 in relation to what the Spanish culture means for the students, it can be said that the semiotic modes appearing in student productions also consist of three types: text, images and audio. These digital posters have a total of 167 words, 67 images and 8 minutes and 46 seconds of audiovisual. Although the Glogster tool has a stronger visual appeal than textual, three of the students have opted to use the written text as a more abundant semiotic mode.
If we compare the results of the two Glogsters, we can see that the relationship between the number of words is almost double. The culture Glogster has double the amount of words compared to the presentation Glogster. As for the number of minutes of the videos, we could say that it is inversely proportional, ie, less words more minutes of oral text, and more words, less spoken text.
For the analysis of the different modes, full attention has been payed to visual mode and various categories have been used for the analysis. Because the aim of study is based on all the issues surrounding the oral language and how it interacts in multimodal communication, only the data from audiovisual documents have been analyzed in depth.

Audiovisual
For the analysis of the audiovisual the categories used range from issues concerning the type of discourse produced -spontaneous oral discourse or read-as issues related to the environment -video alone or with others; video recorded inside or outside-or the cinematic camera effects -visual frame: angle shot; plane angle shot; frontal plane; general plan; deep plane; close-up; intense close up.
As for the verbal oral language, it has been analyzed as followed: volume, voice quality and intonation. When analyzing the intonation, the three functions it possesses have been taken into consideration: the first function, integrative, is the pre-linguistic function of intonation (Cantero, 2002) [11] since its function is not to distinguish different intonation patterns, but to join the sounds of speech together into groups (intonation contours) and thus allow its intelligibility. It is impossible to speak and understand speech without intonation (Cantero, 1991) [12], and therefore, to analyze the students' productions it is essential that their oral discourse possesses pre-linguistic intonation. Along with this, we have also focused on the linguistic function by which the intonation distinguishes different types of sentences from set intonation patterns, ie to differentiate between declarative, interrogative, emphatic sentences etc. Finally, the cases in which tonal variations appear to express emotions would be the third function of the intonation: the expressive function.
For non-verbal language we have set and described gestures, gaze and facial language.
In regards to the video recording of Glogster 1 we have to say it was a mandatory part within this task. Its duration was free and in this sense we have very different results ranging from just one minute to almost three minutes long. In all cases, the recording was spontaneous, with no need to read a text or recite from memory. In all cases, the recording was inside their homes or rooms so that the recording quality was excellent. Also, the plane used by the students was medium close-up, ie showing the bust, from the chest to the head. The choice of this plane and not of the entire body has hindered the visualisation of the nonverbal communication. We could only see the movement of the face and, sometimes, hands.
As for the oral language, we can say that the volume and quality of the voice, as previously said, has always been of good quality. Our students all belong to the B1 level of language learning and this has been shown in their samples of oral language. The intonation was generally very monotonous, with little inflections. The only inflection properly made and worth discussing is the enumeration. The rest of the monologue is a speech heavily influenced by their mother tongue or L1, English. Flow ability, in some cases, has been correct although, in general, being the beginning of the course, they lack enough security and language competence.
Nonverbal communication has been conspicuous owing to its absence. In general, they gesticulated little. The hands have been used, on occasion, to repeat what they were saying. The enumeration has been accompanied by nonverbal communication eg. the movement of fingers. When they talk about things they like and when they talk about themselves, the use of "I", "I like" and "to me" are marked with one or both hands on the chest. Another movement that has accompanied their hands has been "left or right" when describing the location of the photos. As mentioned, the movement has disappeared in many cases almost to immobility. In other cases, however, we must emphasize that the gesture was more natural but this rarely occurred. The gaze and facial language has been the favoured strategy. The smile was also a widely used resource in this regard. It has been noted how they lean themselves from side to side, assumedly, owing to nerves.
Audiovisual documents that make up Glogster 2, also have a duration of more than one minute. The oral text in all cases is spontaneous text and the student is alone in the video, except in one of the Glogster in which the student invited two of his friends to be part of the recording in the background. In the same way, all recordings, with the exception of one, are indoors.
The location of the video tends to be central -in the middle or upper right or left-. No video was found in the bottom part of the Glogster but one audiovisual document was placed in two separate videos, each of them in the central part on either side of the poster.
The most frequent cinematic effects are the frontal plane and the foreground, but a low plane and American plane were also found -this is the case of the student who recorded the audiovisual piece with his two friends. Regarding the verbal oral language, the volume and voice is of good quality in all recordings, without exception.
Three of the students have very good oral speech intonation (they perform many tonal inflections and their intonation fulfils three intonation functions: the pre-linguistic, linguistic and expressive). In contrast, there are three students whose intonation does not respect the tonal inflections and that only meet the first and the second function intonation of the enumerations.
Overall, the students' oral speech is intelligible and a large number of them successfully produce the intonation of enumerations.
The non-verbal language is very rich in all the examples analyzed. Gestures-movements of the hands and head-, facial expressions and gaze -arching eyebrows, grimaces-are significantly present in all audiovisual productions. The functions that the nonverbal language achieves are: compensatory, repetitive and expressive. In the table below a repertoire of gestures and features are shown.

Gesture, Facial Expressions and Gaze
-The head movement accompanying the musicality of their speech -Make discreet grimaces when it encounters obstacles in their speech and fluency-Places hand on chin in order to hesitate -Marks self correction with the forefinger in their speech -Move their shoulders when they are talking in the moments that are making an effort or have difficulty producing their speech in smooth fashion -Move hands when they falter -Move their eyebrows, open their eyes and enhances their expression, all facial movements are consistent with verbal discourse and the difficulties they encounter -It is based on the movement of hands while talking -It helps with the movement of the eyebrows and eyes over his speech. They also make faces when they need time to continue with their speech.

Gesture, Facial Expressions and Gaze
-Indicates the fingers of one hand and the other hand when he lists -He points to the companions when going to present -He waves goodbye when he say goodbye -He waves to start Glogster -When saying the word "dance" he moves his arms as if dancing

Gesture, Facial Expressions and Gaze
-He indicates with his hands when he is saying that it is important -He Mark with the index finger of the importance of what he is saying -With the gaze accentuates what you are saying, for example "it is cool / open your eyes more than normal and expresses that he likes it"

Semiotic Interaction
Having described and analyzed the visual semiotic modes, we will now move to establish the communicative potential which supports the student using semiotic modes from multimodal interaction in which oral language is present. The interaction of different modes has been analyzed following two different relations: a. Semiotic interaction of images and oral text. The functions that arise from the interaction of these modes are: b. Semiotic oral interaction from the oral text and gestures (Martinec, 2004) [13]. In this interaction there are two functions: -Compensatory function (Alfonso, Giralt, 2013) [14], in which the gestures, gaze and facial language deficiencies outweigh oral audiovisual speech.
-Repetitive function, in which the gestures, gaze and facial language repeat the same idea and oral discourse meaning.In analyzing the different semiotic modes that appear in the different glogsters in relation to oral language, interaction has been found between them, so that there is a semantic expansion which enriches the communication process.

Oral Text and Image
The relationship and interaction of these two semiotic modes are based on the features that have been discussed above: -Referential function. In this function, the images illustrate or reiterate what is said and explained in the audiovisual recording. Of all the analyzed Glogsters, there is only one that does not fulfil this function; the rest of the oral text modes and images have a relationship with a referential purpose.
-Textual function. From this function, the interaction of the two semiotic modes pursues an expanded outreach and communication of meaning. Again, in all of the Glogsters except for one-the same one which also lacked referential function-semiotic interaction of both modes perform a textual function.
-Persuasive function. This function is the one that is less present in the Glogsters discussed. It is possible that credibility is already conferred in what the students are saying owing to the fact that they speak personally in a video recording. It has been considered that the only Glogster that fulfills this function is the one in which the student invited two of his friends to participate in the video. With the inclusion of two friends in the video, the student lends credence to the theme of friendship, a very important issue in their Glogster. -Rhetoric function. This feature has an aesthetic purpose, which is in all the Glogsters. In fact, we found a Glogster in which the interaction of oral text and image mode only has the rhetorical function. All compositions used aesthetic resources such as colors, different fonts and font size, AutoShapes, clipart, ornate text boxes, etc.. Students design their entire Glogster attractively to entice, excite or please the reception of it.

Oral Text and Gestures
Another interaction that occurs between semiotic modes appears in the audiovisual document. During the students' oral speech, gestures are another way of communicating meaning. This relationship that is present in all the posters that speak of culture has been duly noted after our analysis, either with a compensatory function or a repetitive function. However, in the presentation glogsters there is a striking absence of this type of nonverbal communication.
The first of these features is helpful in students' oral production. Gestures, facial language and the gaze all serve as a support for the shortcomings of students' oral communication. Likewise, students use hands to say hello or goodbye, to reiterate the significance of what they are saying and to emphasize or accompany an enumeration, for example. This is a reality in face to face communication in which two parties exchange information. What is relevant in our case is that the two semiotic modes interact functioning as a compensatory strategy in a digital medium.

Mahara Portfolio Views
To analyze the composition and construction of the portfolios' first view, the item "construction of the views", established under headings designed by the Group d'Innovation docent DIDAL (Universitat de Barcelona) have been followed in the analysis. It is argued that multimodal diversity is also present in the way that they design their view in the digital portfolio. In most cases, students show diversity in the use of semiotic modes and they combine them correctly, getting to use each of the different modes of the discourse functions: introduce themselves, inform, contextualize, say goodbye; illustrate.
At the same time, in all the views that they have designed and constructed, a sense of proportion is appreciated, placing more emphasis on the visual in some cases and in the textual in others. We also found that some of the compositions -see Figure 1 and 2-show an arrangement of different elements for constructing a meaning and observe hierarchies between visual elements. In Figure 1, we can see the first view of a Mahara portfolio of a student. At first glance, we see that the organizational arrangement of text and images is in two columns. This view consists of the Glogster 1 and 2 and a personal reflection on each of them.
In contrast, in Figure 2, we see how the organizational structure of the view is composed of three columns. Besides the images of the Glogster, the student has added the videos, web links and a personal address. As we see, there is almost no text commenting on the images or reflecting on the task.

Discussion
The varied types of semiotic modes that appears in the analyzed data of the Glogsters has a slight relationship between the amount of text and the number of images as they always find more images than more text. This relationship is understandable if one considers that written text and images often have an illustrative or expansive relationship, and therefore, more text will be illustrated with more pictures and more pictures are going to need more text to expand its meaning. By contrast, an inverse relationship occurs between written text modes and oral text, since, more written text, less oral text and more oral text, less written text.
Given these results, it can be said that the existential and experiential issues of students take their full significance when they interact with other semiotic modes. The most used mode to convey meaning related to the experiences of students tends to be the oral mode. Significantly, most of the students placed the video in the center of the composition of the Glogsters making it the most important component. Thus, the hierarchy of the composition of the elements of the glogsters reinforces this idea. The videos are central in 50% of the Glogsters analyzed, in 25% a text box is used and in the other 25% pictures of either culture or themes as personal as the family are used.
The oral mode combines firstly with non verbal language, but it also does it, if possible, with pictures and written text.
It is believed that the use of multimodality serves on the one hand, to support the students' oral production, especially for those who have more difficulty and less oral communication skills, as already pointed out by Ciekanski, M&Chanier, T [15]. Proofs of this are the interactive processes of the oral language and the gestures. The oral language that appears in the audiovisual generally complements its meaning with gestures, facial language and gaze. This interaction allows it to reiterate, emphasize or compensate the meaning of what is being communicated. Although this has been most evident in the Glogster 2, it has also occurred in the Glogster 1.
On the other hand, the multimodality and semiotic interaction that occurs with the different modes that make up the Glogsters allow semantic expansion which enriches the communication process. This interaction takes place between all modes, and in light of the results discussed, when the oral language interacts with the images it conducts referential and textual functions.
Finally, commenting on the discussion of the obtained results from the analysis of the view of the digital portfolios, students have had no problem in using semiotic modes in their compositions of the digital portfolios. Most of them have a digital competence that has allowed them to create a multimodal discourse, with a wider meaning or a superior sophistication in their compositions, as appropriate. This fact has already been observed in the compositions of the Glogsters, which is also evident in the composition of the analyzed view of their portfolios.

Conclusions
The discourse analysis and multimodal communication of the compositions of the posters and the digital portfolios of SFL students has enabled us to see how the semiotic interaction that occurs is a process that enriches and enhances communication processes present in digital media when used for learning a foreign language.
The use of different modes and their combination can be considered multimodal strategies, understanding multimodal strategies as scaffolding strategies that enrich the repertoire of the learner's communication resources. Each student has used modes that have been adjusted to their needs and skills. From this perspective, it is argued that students develop their strategic competence to achieve the communication of a global significance, using multimodality and combining the different skills they possess.
The semiotic interaction that occurs between the oral and the other modes, reflects the potential of multimodal interaction, especially when it works as a complement to oral language communication gaps in learners of SFL, ie when functioning as compensatory function.
It is necessary to point out that the semiotic interaction among modes stimulates development of oral language skills when spoken language is combined with any mode except written language. The relationships that occurs between written text and oral text is that when written text is more predominant there is less oral text used.
In short, we believe that the oral language is an other mode that integrates in the multimodal discourse in a digital media for the learning of SFL. The interaction between semiotic modes enhances the communication processes that are carried out through a digital medium for the learning of SFL.