Blog
IQP Week 2
This week the Know and Open team decided to officially join forces, we have split into two teams in order to best make use of our combined resources. One team is currently working on writing a combined methodology by taking applicable parts from each teams existing methodology making them flow together. The other team is busy coding, we are continuing to develop the VPC Data Sandbox as well as starting the VPC Data site. Because of our increased team size we are able to work on multiple parts of our project in parallel much easier than was previously possible. |
IQP Week 1 Progress
Open and Know team have been collaborating for the first week on getting our sandbox setup working for generating interactive visuals online. We've established a code base and basic navigation around the site, and our plans for the coming week are to now start implementing the visual editor interfaces, and improve on the interface. The code for this project can be found at: https://github.com/gr-b/sandbox   |
Week 6 Status Update
The 3.0pen team has been hard at work this week finishing up our first full draft of our proposal. The full proposal can be found here. The final presentation draft can be found here. We have revised our objectives, and feel confident about our ability to accomplish these during our term in Venice:
As part of our full draft, we also fleshed out and completed our methodology section. An outline of our sub-sections is below: 3.1 Validating and integrating VPC data for publication in print and online 3.1.1 Assessing the completeness and correctness of existing data 3.1.2 Collecting missing data 3.1.3 Adding data to Firebase 3.2 Analyzing and visualizing data about selected topics 3.2.1 Determining the most appropriate visuals for each data set 3.2.2 Coding the interactive visuals 3.3 Make the VPC data accessible to the public via an online platform 3.3.1 Designing the layout of the VPC 3.0 Website 3.3.2 Coding the 3.0 website templates 3.3.3 Connecting the templates to the data sets 3.3.4 Publishing the Venice 3.0 Website We've also continued our analysis of the City Knowledge Console data files. Below is a chart displaying the additions of store data over time, followed by a screenshot of altar data on an interactive map we created:   |
Week 5 Blog Post
This week the team did more research into other IQP projects that we can learn from, in addition to the 4 projects that we already found. We added these to the background section talking about the past work at the VPC. We also did some analysis of the existing data sets, particularly about the CSV data that Nicola gave us. The data stored in the database is split between 14 tables, corresponding to a variety of different topics including churches, city islands, and stores. Across all of the tables, there are nearly 10000 rows of data, and most of the tables have 30 or more columns. An average table consists of about 49000 fields of data, and is nearly half a megabyte in size. The largest table has nearly 5000 lines, over 100000 fields, and is over 2 megabytes large. Unfortunately, a sizable chunk of each of these tables has not been filled in with data. Overall, around 25% of the total fields are either empty or filled with null data. One of the tables is missing nearly 75% of its data. |
| Attachments:
CSV Metadata.txt
VPC Past Projects and Venipedia
This week, the team did a little more digging into what the past projects at the Venice Project Center have done. We discovered that some of the accomplishments over the years were the creation of infographics and maps, visualizations and widgets for a VPC dashboard, a VPC smartphone application, a new Venice site containing datasets of selected topics, the City Knowledge Console tool that uploads all collected data about a topic and allows access to it through the VPC website, and finally, several interactive visualizations of the VPC data. In addition, we looked at some statistics about the current Venipedia site that numerous past VPC projects worked on. Venipedia is a wiki-based website that provides an abundance of original information, including detailed facts and statistics, about Venice that cannot be found elsewhere. Currently, the site has 27,120 total pages of information as well as 19,312 different articles. Each of the topics can have numerous pages. For example, the bridges category consists of 433 individual pages, one for each bridge in Venice. In addition, the site contains 706 uploaded files, which include various images, graphs, and maps. |
Global Data Management Problem
The issue of large scale data organization is a prevalent problem worldwide. Especially with the increasing amounts of data and information flow in recent years, managing such large amounts of data and allowing the public to access and use this data effectively is difficult. The goal of making data open and accessible is so that people can use this data for other purposes. This can only be done if the data is displayed it in a way that will be easy for people to view and make sense of. Organizations around the world are addressing this issue by developing initiatives to give better open data access as well as improve on visualizations. Similarly, we are helping the VPC address this data management problem by assisting them in making the data they’ve collected over the past 30 years more accessible to the public and make it more easily useable to organizations (such as local Venetian startups), to future IQP projects, and to anyone else who could benefit from using the data. For example, the EU estimates that 25,000 jobs could be created by making data more accessible (Benefits of Open Data, 2017). Below are some of the other benefits of open data: (image also from the European Data Portal)
"Benefits of Open Data." . Accessed Sept 17, 2017. https://www.europeandataportal.eu/en/using-data/benefits-of-open-data. |
Past work of the Project Center
In the years the VPC has been active, there have been a few projects that worked on managing the data gathered from other Venice IQPs.  The first data management project created a site to aggregate and visualize the data from the previous twenty years of projects (Brunelli, Orsi & Sargent 2010). This was the first time that there was a project with a significant effort dedicated solely to organizing the data of the VPC. This created the Venice 2.0 site, which is the previous version of the Venice 2.5 site that currently exists as of the writing of this post.  That same year, a group worked on the original Venipedia. They created the site because there was no easy way to access the data stores of the VPC. They modeled their site as a wiki to allow anyone to be able to contribute (Finelli, O'Brien & Scannell 2010). Two years after the original Venipedia project, a group improved Venipedia by migrating it to a new location and filtering its content. Due to the open nature of the wiki, the site was “plagued by spam” and riddled with errors, especially due to contributors not following submission guidelines (Bobell, McCarthy, Poganski & Tsiros 2012). Though it is invaluable to the Venice Project Center, this site shows one of the potential weaknesses of Open Data collaboration: standards enforcement.  Sources: Bobell, Patrick, Lauren McCarthy, David Poganski, and Alyssa Tsiros. "Venipedia."Worcester Polytechnic Institute, 2012. https://drive.google.com/file/d/0B8nycmDtoovaRW9BYjRKNTRrZmM/view. Bremner, Joseph, Austen Couvertier, and Frank Egan. "Venice Big Data."Worcester Polytechnic Institute, 2016. Brunelli, John, Edward Orsi, and Amanda Sargent. "Showcasing Twenty Years of Venice Project Center Results using Interactive Online Infographics."Worcester Polytechnic Institute, 2010a. https://web.wpi.edu/Pubs/E-project/Available/E-project-121710-153620/unrestricted/VE09-40_FinalReport_10222010.pdf. Finelli, Thomas, Catherine O'Brien, and Kevin Scannell. "Venipedia."Worcester Polytechnic Institute, 2010. https://3089019a-a-62cb3a1a-s-sites.googlegroups.com/site/ve10tude/project-management/files/FinalReport.pdf. |
The Open Data Movement
The VPC has advised many projects in the past that deal with data collection or publishing. Most of the objectives of these projects align with the objectives of a larger movement called the Open Data Movement. The Open Data Movement is a movement that aims to publish local, regional, and national data, with the hopes that people / organizations will use that data for their benefit. The types of data being published can be anything that might be of use to society, ranging from weather, traffic, tourist information, business, political, and geographical data (Gurstein, Michael B. 2011).  The benefits of making data open can be broken down into three sub-categories: Political / social, economic, and technical. The political benefits range from transparency of government operations, accountability, insights into the public sector, and equal access of data for everyone, regardless of social status. Economic benefits, such as availability of info to investors, and stimulation of competitiveness, could be of more importance for the VPC with the decline of local business and jobs in venice. Finally, the technical benefits are that by reusing data (not having to recollect data someone else might have already collected), optimizations can be made to administrative and research processes, resulting in time and money saved (The EU estimates they can save billions every year through open data related optimizations) (Janssen, Marijn, Yannis Charalabidis, and Anneke Zuiderwijk 2012; “Benefits of Open Data”). There are many new companies that are working to advance the Open Data Movements objectives, a good example being a UK-based firm called Arup. Arup is an independent company consisting of designers, planners, engineers, and technical experts, with the mission of using open data to help solve the world’s problems. The projects they’ve worked on address questions such as: How do you preserve an architectural icon for future generations?, How do you sustainably and competitively shape a city’s port to fulfil its role at the centre of the economy?, and How do you breathe new life into one of London’s busiest streetscapes? (Arup). By leveraging traffic, planning, natural hazards and other topics, Arup is able to “deliver more efficient services and help mitigate against risks, for example of natural disasters” (Charlotte Jee). With these benefits also come some barriers that could potentially hinder the open data movement’s success. The primary barrier to be aware of is the risk that users won’t have enough incentive or awareness of the available data, won’t utilize it, and won’t benefit from its publishing. Along those lines, information quality is critical to ensure proper extrapolation, and maintain trust by the parties using the data. Some of the technical barriers typically encountered are lack of supporting infrastructure, fragmentation of data, and legacy data formats (issues the VPC has communicated are of importance to address during our IQP). It’s crucial to recognize these barriers because the same barriers that apply to the open data movement will most likely be issues for the VPC in it’s mission to create a hub for Venetian open data (Janssen, Marijn, Yannis Charalabidis, and Anneke Zuiderwijk 2012). Citations: "Arup." . Accessed September 19th, 2017. https://www.arup.com. "Benefits of Open Data." . Accessed Sept 17, 2017. https://www.europeandataportal.eu/en/using-data/benefits-of-open-data. Charlotte Jee. "19 Innovative UK Companies using Open Data" . Accessed September 19th, 2017. https://www.techworld.com/picture-gallery/startups/-innovative-uk-companies-using-open-data-3613884/. Gurstein, Michael B. "Open Data: Empowering the Empowered Or Effective Data use for Everyone?" First Monday 16, no. 2 (2011). Janssen, Marijn, Yannis Charalabidis, and Anneke Zuiderwijk. "Benefits, Adoption Barriers and Myths of Open Data and Open Government." Information Systems Management 29, no. 4 (2012): 258-268. |
1-10 of 12