A trip into the mind of Wajdi

Being a part of the Y generation, I grew up listening to various types and genres of music, from Rap to RnB, to Rock and electronic music. I was personally choked to hear about the death of the lead singer of the Linkin Park band. Their music was and still is popular until today. The news was as choking to me as to anyone else who knew that band. After knowing about the news, I had a discussion about this tragedy , and a question popped up about the fact that rock band members mostly die of suicide or drug over-dose. I wasn't really sure that this information is true, so I really couldn't have a real argument to say that it is maybe (or not ) the most commun reason of death for Rock stars. And since I love playing with data, I decided to fact-check the hypothesis stating that:  "The most commun cause of death among rock band members is suicide" . To start, we have to get  some data! and since there is no available data-set to analyse , I had to be creativ...

As I have mentioned before, I have started giving a workshop at the Higher School of Statistics and Data Analysis; The subject is the economic and statistical analysis of the Tunisian Economy. I share here this week's requirements. Hello Team, I hope you are getting excited to start working with the Data! Starting from this week, we will not have the usual presentations, as this methods seems to be a time-waster. I will look at each group's work individually and have a workshop-methodology in reviewing your work each week. (this does not mean that you are not required to present your work! I expect the same quality and format of presentations each week, and you all have to be present at the class and work on your data  and show me the problems you have if there are any)

As I have mentioned in a previous post, I will be giving a couple of statistical workshops and courses at the Higher School of Statistics and Data Analysis of Tunis this year. The second course I am giving is about Market Analysis and Customer Segmentation Using Machine Learning Algorithms. Bellow the project description and some details about the goal of the work we will do this year.

Following the Statistical Analysis Software course I gave in the first semester of this year, I have started giving a couple of classes to freshman and sophomore statisticians . The first course is a statistical workshop during the second semester that will be focused on economical and statistical analysis of the Tunisian Economy.

After we have explained in the previous parts, how to set up the access to Twitter's API and how to import tweets with a simple R command, in this third part we will try to organize and clean the data we have imported in good reusable format. So let's pick up from where we left off , after finishing the authentication process, now we want to import certain tweets. the command  searchTwitter  from the package twitteR will do the job for us.  we need to create a new object that will contain the data we are about to import and give it the result of our  searchTwitter function. In our example, we will be querying Twitter for the tweets containing the hashtag #Tunisia , we will import 1000 English tweets. and we will have a preview of the imported data with the head()  function:

F irst part of this series of posts. This is the second part of a series of posts that treat the subject of strings and text manipulation in R. Every time some functions are picked with basic examples and shared here. I hope that at the end of this series, I'll have a good archive of lessons from beginners level to expert level. The points we will see in this part 2 are: character encoding  length of a string  lowercase and uppercase conversion  basic string comparison  concatenating strings extracting sub-strings   character encoding 

Twitter is a magnificent source of very interesting data about the world, trends, products, celebrities, current and past events..etc This is why I  have been interested in analyzing and working in Twitter data for a long time. In part one we learned how to set up an application and get some codes and key's to use later one. In this part of the tutorial, we will look at the ways of searching and importing data from Twitter. The authentication process is very easy: it has mainly  2 parts: - Entering the keys and secrets you had from Twitter - Submitting them for authentication and get access to Twitter's API.

I wanted to do this quick tutorial because of an observation I made while working with some New R users who struggle with operations that does not involve numbers, operations and statistical calculations. Handling text, images, files of all formats, are operations made possible within R via its numerous packages. This time I started with basic strings, and I will probably add to this series later. We will learn 4 new functions: grep grepl These 2 function searches for matches of a given string variable, within each element of a character vector. the only difference between them is the output. The first one gives the position of the string that matches the search, the second one gives a logical result of TRUE or FALSE for all the strings of the vector. gsub This function performs a substitution of a given string by an other in all the strings given as input for the function. str_replace This function replaces first occurrence of a matched pattern in a string...

All "Smart" Businesses are looking to understand the social media trends , to analyze the massive amount of public data available online and to make insightful decisions based on these analysis. Source :http://www.slashgear.com/twitter-data-grants-introduced-to-offer-select-institutes-data-trove-05315867/ For any statistician or future data scientist freshly graduating out of the university, it is very important to be able to have certain skills with the statistical modeling and mathematical knowledge. In this series of posts, I will detail the necessary steps for that you will need to access Twitter, import data, clean it and analyze it and have a conclusion based on the data you have extracted. It will be a simple step-by-step tutorial if you'd like to call it that way. This series of posts is destined to students, currently taking data science classes, and to anyone interested in R language and social media in particular . I've been asked to make ...

Hello World! The topic of today's article is databases. As Data Scientists and Statisticians work with data everyday, they wont actually use that 50 lines text file data-sets provided by teachers in the statistical analysis courses in a real-world applications. Statisticians work with massive amounts of data, whether this data is stored in flat-files or in databases, the size of the analyzed data will definitely be more then a couple of hundred records. Thus the need for a way to extract data from large tables stored in databases in a simple, intuitive way. Whats is SQL ? SQL means  Structured Query Language . For me, it has always been the "Simple Query Language". I've always used the term  "Simple"  to describe the simplicity of learning and using basic sql functions.

I was asked by new students in the Statistics and Data Analysis School of Tunis (and by others friends) about ways to plot densities and the best software to do that. I will try to give some examples with  R software on how to plot density . If you don't know what R is, you can take a look at my old article that explains that very well with some good R learning resources . ( here in French ). You know that densities are all about random data and the first thing that comes in mind is histograms. To start this task, we will first set the data sample that we will work with which should be random, and then we will start plotting some nice graphics with the ggplot2 library. Basic things :Histogram and density plots: regular histogram where is the mean! density plot Overlay of density and histogram

C'est quoi "R" ? (pas la lettre, mais le langage! ) Mon premier post est dédié à un sujet que j'aime discuter souvent: R . Pour commencer, ma première rencontre avec R était à l'école supérieure de la statistique et de l'analyse de l'information de Tunis. Où ils enseignent un cours du logiciel R, et 80% des cours pratiques de statistique sont démontrés et réalisés avec R ( Nous avons eu des cours de SAS, Eviews et Excel aussi). C'était donc dans le cadre des études d'ingénierie statistique et analyse de l'information que j'ai fait connaissance du grand logiciel et langage R. C'est quoi R donc?! R est un langage et un environnement de programmation  statistique et graphique. Il est inspiré du langage S. Ainsi on peut considéré R comme une sorte d'implémentation du langage S. Parmi les points forts de R c'est la simplicité de sa programmation, la qualité des graphiques qu'il génère et la multifonctionnalité associ...